IntentSharpby C2Consultants Read the quick start
Menu
Documentation
Table of Contents

Documents and edits

Rule files, intent files, the settings file and the provider file are documents. A document store keeps each one as text, with its versions. An edit proposes a new version: the validators check it before it is stored, and a writer decides whether it runs. A person, your code and a model all edit through the same seam.

The store

IDocumentStore keeps the text and its versions. Two stores ship in the core:

  • InMemoryDocumentStore keeps everything in memory, for tests and a process that does not restart.
  • FolderDocumentStore keeps the working copies in a folder and their versions in a state folder inside it, .intentsharp, unless StateRoot names another. With WatchForChanges, on by default, it reads the folder again after a file is created, changed, renamed or deleted, once the folder has been quiet for DebounceInterval, 250 ms unless set.
// "policy" holds the documents: rule files, intent files, the settings file and the provider file. Their
// versions are kept in policy/.intentsharp, or the folder StateRoot names.
using var store = new FolderDocumentStore("policy", new FolderDocumentStoreOptions
{
    MaxDocumentBytes = 1024 * 1024,   // a larger working copy is reported and not read (4 MB unless set)
    MaxVersionsPerDocument = 50,      // 100 unless set
});
using var client = new IntentClientBuilder()
    .UseProviderOptions(store)
    .WithDocumentStore(store)
    .AddDocumentValidator(new IntentDocumentValidator())   // checks edits of intent files
    .Configure(o => o.MaxEditBytes = 256 * 1024)            // a larger edit is refused unread
    .Build();

Under dependency injection, services.AddDocumentStore("policy", o => ...) registers the store, and services.AddIntentCatalog(store) registers the intent file validator with it.

A document's id is its path in the store, with forward slashes, such as intents/Orders.Priority.intent.yaml. Ids compare ordinally, so on a file system that ignores case, two ids that differ only by case share one working copy. The folder store refuses a path that would leave the folder or that Windows would read as another file, and does not follow a folder link out of it. Its lock covers one host: applications on several hosts that share documents need a store of their own on shared storage.

Write access to the working folder and the state folder is full trust. Whoever can write the working folder can change any rule file, settings file or the provider file: the validators check the new version, but an edit made to a working copy goes to no writer. Whoever can write the state folder can add a version that the next start runs.

The intent, rule and provider files may go into source control, and are reviewed like code. The .intentsharp folder the store writes beside them stays out of source control, and out of backups that people without access to your keys can read: it keeps the text of every stored version, a rejected one included, so a key pasted into the provider file stays there.

Versions, history and revert

Every stored version keeps its sequence, its author, its reason and the version it was based on, and exactly one version of a document runs. The examples on this page edit this intent file:

name: Orders.Priority
version: 1
owner: order-desk
intent: Rate how soon this order must ship, from 1 (this month) to 5 (today).
shape: int
rules:
  minimum: 1
  maximum: 5

// Every stored version of the document, rejected ones included, oldest first.
await foreach (DocumentVersion version in store.GetHistoryAsync(id))
    Console.WriteLine($"{version.Sequence}: {version.Author.Kind} {version.Author.Label}{(version.IsRunning ? " (running)" : "")}");

// Back to the version that ran before this one. The history keeps both, and the next call runs this one.
DocumentVersion running = await store.RevertAsync(id);
Console.WriteLine($"Running: version {running.Sequence}");

RevertAsync returns to the version that ran before the current one, which is not always the previous sequence: after activating versions 1, 3 and then 2, a revert returns to 3. ActivateAsync(id, sequence) makes a stored version the running one. Both apply to the next call, with no restart. The folder store keeps up to MaxVersionsPerDocument versions of each document, 100 unless set.

The edit seam

An edit is a ProposedEdit: its kind, its document, its content, its author and why. client.Edits.SubmitAsync checks it and passes it to the edit writer:

var edit = new ProposedEdit
{
    Kind = EditKind.Document,
    Document = new DocumentId("intents/Orders.Priority.intent.yaml"),
    Content = proposedText,
    Author = new EditAuthor { Kind = AuthorKind.Model, Label = "assistant" },
    Reason = "Treat a stopped production line as the most urgent.",
    // BasedOn: the version the text was written from. Left null, it is the version running now.
};

EditOutcome outcome = await client.Edits.SubmitAsync(edit);
switch (outcome.Status)
{
    case EditStatus.Applied:           // stored, and running from the next call
    case EditStatus.SavedNotRunning:   // stored as a candidate; the running version stays
        Console.WriteLine($"{outcome.Status}: version {outcome.Version!.Sequence}");
        break;
    default:                           // Refused, Rejected or Conflict: nothing was stored
        Console.WriteLine(outcome.Status);
        foreach (var reason in outcome.Reasons) Console.WriteLine(reason);
        break;
}
if (outcome.Impact.WidensSecurity)
    Console.WriteLine("Widens: " + string.Join(", ", outcome.Impact.WidenedSubjects));
Status Meaning
Applied Stored as a new version, and that version runs from the next call.
SavedNotRunning Stored as a candidate. The version before it keeps running until the candidate is activated.
Refused IntentSharp refused it: it is larger than the limit, does not load, no validator handles its document, or the store refused to save it. Nothing was stored.
Rejected A writer of your own declined it. Nothing was stored.
Conflict The document changed since the version the edit was based on. Nothing was stored: read the running version, merge, and submit again.

The validators of rule files, the settings file and the provider file come with the features that read them. For intent files, add IntentDocumentValidator, as above. An edit of a document that no validator handles is refused.

Edits apply by default

IntentSharp applies an edit by default, a model's edit included: its writer, ApplyingEditWriter, stores the version and makes it the running one. An edit that allows more than the running version allows, or removes a security limit, is applied and flagged: EditImpact.WidensSecurity is true, and WidenedSubjects names what it widens. An edit that only narrows is not flagged, and intent files hold no security item, so an edit of one is never flagged. The Guardrails and Provider secrets guides list what widens in a rule file and in the provider file.

To review edits before they run, wrap the writer. This one holds every edit a model proposes as a version that does not run:

/// <summary>Stores every edit a model proposes as a version that does not run, for a person to activate.</summary>
public sealed class HoldModelEdits(IEditWriter inner, IDocumentStore store) : IEditWriter
{
    public async ValueTask<EditOutcome> SubmitAsync(ProposedEdit edit, CancellationToken ct = default)
    {
        if (edit.Kind != EditKind.Document || edit.Document is not { } document || edit.Author.Kind != AuthorKind.Model)
            return await inner.SubmitAsync(edit, ct);

        var version = await store.SaveAsync(new DocumentSave
        {
            Id = document,
            Text = edit.Content,
            Author = edit.Author,
            Reason = edit.Reason,
            BasedOn = edit.BasedOn,
            Activate = false,
        }, ct);
        return new EditOutcome { Status = EditStatus.SavedNotRunning, Version = version, Impact = edit.Impact, Message = "Held for review." };
    }
}

var client = new IntentClientBuilder()
    .UseProviderOptions(store)
    .WithDocumentStore(store)
    .AddDocumentValidator(new IntentDocumentValidator())
    .WithEditWriter(inner => new HoldModelEdits(inner, store))
    .Build();

Under dependency injection, wrap it with DecorateEditWriter. A wrap needs a document store: a client with a wrap and no store is not built. IntentSharp ships no review queue: a held edit is a stored version, and it runs when you activate it. A wrapper cannot hide the validators' assessment, which every outcome carries.

On scripted edits of the intent file: a model's edit with the default writer, an edit based on a version that no longer runs, an edit that does not load, an edit over MaxEditBytes, the history and a revert, and the model's edit held and then activated:

A model's edit, with the default writer:
  Applied: version 2 is running
An edit based on version 1, written while version 2 began running:
  Conflict
An edit that does not load:
  Refused
  - intents/Orders.Priority.intent.yaml:4: The file is not valid YAML: While parsing a flow sequence, did not find expected ',' or ']'.
An edit larger than MaxEditBytes, 16 KB here:
  Refused
  - intents/Orders.Priority.intent.yaml: The edit's content is 20,216 bytes, more than the limit of 16,384 bytes, so it is refused unread.
The history, then a revert:
  1: External
  2: Model assistant (running)
  Running: version 1
The model's edit again, with HoldModelEdits:
  SavedNotRunning: version 3; version 1 keeps running
After a person activates it:
  Running: version 3

Size limits

A document is at most 4 MB (4,194,304 bytes) by default. A working copy larger than the folder store's MaxDocumentBytes is reported in Problems with its size and is not read, so the version before it keeps running, and a save of a larger text throws ArgumentException. The edit seam refuses a larger edit before any validator reads it: its limit is IntentClientOptions.MaxEditBytes, which follows the folder store's limit when it is not set and is 4 MB for any other store. Both limits are set in code, and no document can raise them. A store of your own that wraps a folder store is not a folder store to the client, so set MaxEditBytes to at most the folder store's limit there.

Side-by-side runs

A candidate version can run beside the running one before anyone activates it. Side-by-side runs are experimental: a call is a compiler error, INTENTSHARP002, until you suppress it. An intent trial runs each sampled call of an intent in the document a second time, with the candidate's definition and the same input, and compares the two answers:

#pragma warning disable INTENTSHARP002   // side-by-side runs are experimental
// The candidate is stored, not running: version 1 keeps answering.
DocumentVersion candidate = await store.SaveAsync(new DocumentSave
{
    Id = PriorityFile,
    Text = candidateText,
    Author = new EditAuthor { Kind = AuthorKind.Developer, Label = "dana" },
    Reason = "A stopped production line is always 5.",
    Activate = false,
});

// Each sampled call of an intent in the file also runs the candidate on the same input. The caller gets the
// running version's answer; the report counts how often the two agreed.
await client.Trials.StartAsync(new IntentTrial { Document = PriorityFile, Sequence = candidate.Sequence, SampleEvery = 1 });
foreach (var email in emails)
    await runtime.RunAsync<int>("Orders.Priority", email);
TrialReport report = (await client.Trials.GetReportAsync(PriorityFile))!;
await client.Trials.StopAsync(PriorityFile);
Console.WriteLine($"Sampled {report.Sampled} of {report.Calls} calls: {report.Agreed} agreed, {report.Differed} differed");

// A trial promotes nothing. When the candidate has earned it, a person activates it.
await store.ActivateAsync(PriorityFile, candidate.Sequence);
#pragma warning restore INTENTSHARP002

Running before the trial: version 1
Sampled 3 of 3 calls: 2 agreed, 1 differed
Model calls: 6, three for the running version and three for the candidate
Running after ActivateAsync: version 2
  • The caller always gets the running version's answer. The candidate's model calls count in the result's usage and in the client's totals.
  • The intent runtime runs the candidate of a definition it loaded from a document store, such as a DocumentIntentCatalog, under the guardrail sets of both versions, so the candidate never runs under weaker rules.
  • With a document store on the client, a trial's counts are also kept in the store's state, so a trial started again, after a restart or in another process, continues them. The counts are advisory.
  • Rule files can run a candidate in shadow with GuardrailOptions.Shadows: its rules are evaluated detect-only beside the running version's, and a rule whose verdict differs is reported in the decision's ShadowDifferences. Only the running version decides.
  • A trial or a shadow never promotes a candidate: activating it is yours to do, with ActivateAsync. The settings file and the provider file have activate and revert only.

What is kept

The history keeps the text of every stored version, a rejected one included. A working copy that does not load is kept as a rejected version and never runs, so a key pasted into the provider file stays in the history. An edit submitted through the seam that does not load is refused and never stored.

A reply the Reply checkpoint refuses is not kept. Its attempt's RawOutput, the result's RawOutput when it was the last reply, and the memory entry hold a marker that names the rule; the next attempt is sent the rule's message without the reply, and the request cache does not store it. When the last reply was refused, the decision in ReplyGuardrails names each failed rule, and not the value:

A reply with markup, under the set replies:
  Attempt 1 RawOutput: [reply withheld: the Reply checkpoint did not allow this reply, so its text is not kept; rule plain-text on reply.strings (guardrail set "replies")]
  Attempt 1 issue: [Guardrail] attempt 1: Write plain text, with no markup.
  Attempt 2 passed; Success: true

Every other reply is kept as the model wrote it, in RawOutput and in each attempt's RawOutput, a reply that failed verification included. Treat both as untrusted text.