Input contracts
A call can only use what it is sent. When an intent needs a fact that its input leaves out, a model either says so or makes something up, and an invented value can pass your type. Most problems in a chain of calls are of this kind: a ranking that is not told the delivery mode it ranks for, or a proposal that must cite files but is sent their text without their names. The fix is in the input and the result type, not in the model: declare what each operation needs, give the model a place to say what is missing, and check what it must not invent.
Declare what each operation needs
Give each operation an input type of its own, with a member for every fact its intent uses, and a result type that says what may stay empty:
/// <summary>One way to deliver a job's materials.</summary>
public sealed record DeliveryOption(string Name, double Cost, int Days);
/// <summary>What the ranking needs: the job's delivery mode, which decides what "best" means, and the options.</summary>
public sealed record RankingInput(string? DeliveryMode, IReadOnlyList<DeliveryOption> Options);
/// <summary>The options in order, or no order and the facts that were missing.</summary>
public sealed record Ranking(IReadOnlyList<string>? Order, IReadOnlyList<string> Missing);
- The input goes to the model as JSON, member by member. A fact the type names is a fact the model receives; one it does not name is not sent.
DeliveryModeis nullable because the application may not know it yet. A null member still reaches the model, asnull, so the model can see that it is missing.- Where an operation cannot run without a fact, make the member non-nullable and check it in your code before the call.
Let the model say what is missing
The model fills every member of the result type, so a member it cannot fill from the input is invented unless the type lets it stay empty. Give the answer a nullable member and add a list for what is missing, and say in the intent that the model may use them:
AIResult<Ranking> ranked = await input.AIAsync<Ranking>(client,
"Rank the delivery options, best first, for the job's delivery mode. If the input does not state a " +
"delivery mode, return no order and list what is missing. Do not assume one.");
if (!ranked.Success) return null;
if (ranked.Value!.Order is null)
Console.WriteLine($"Ask for: {string.Join(", ", ranked.Value.Missing)}"); // the model declined; ask, do not guess
Success is true when the model declines this way: the answer met your type. Check the empty member, and ask for the
fact rather than guess it. Whether the model declines or guesses is the model's judgment, which the intent guides;
IntentSharp checks the answer's shape, not what the model knew. What your code can check, it should.
Check what the model must not invent
Send the names with the texts, and refuse an answer that cites any other name:
/// <summary>A source file: its name, which a proposal cites, and its text.</summary>
public sealed record SourceFile(string Name, string Text);
/// <summary>A proposal, the file each statement comes from, and what it needed but was not given.</summary>
public sealed record Proposal(string Summary, IReadOnlyList<Citation> Citations, IReadOnlyList<string> Missing);
/// <summary>One statement of a proposal and the file it comes from.</summary>
public sealed record Citation(string File, string Statement);
// The names travel with the texts, so the model can cite them, and a citation of any other name fails the check.
var names = files.Select(f => f.Name).ToHashSet(StringComparer.Ordinal);
var options = new AIRequestOptions().Validate<Proposal>(proposal => proposal.Citations
.Where(c => !names.Contains(c.File))
.Select(c => $"Cite only the files in the input; {c.File} is not one of them."));
AIResult<Proposal> proposal = await files.AIAsync<Proposal>(client,
"Write a one-paragraph proposal from these files, and cite the file each statement comes from. " +
"If a statement needs a file that is not here, list it under missing instead.", options);
A citation of a file the input does not hold fails the check, and the model is sent the message with the next attempt. Keep anything you would not send to the model out of the message.
The same contract in an intent file
An intent file states the result's shape as a JSON schema. A property whose type lists "null" may stay empty, and
required makes the model write it either way:
name: Delivery.RankOptions
version: 1
owner: estimating
intent: >
Rank the delivery options, best first, for the job's delivery mode.
If the input does not state a delivery mode, return no order and list what is missing.
Do not assume one.
shape:
kind: schema
schema:
type: object
properties:
order:
type: [array, "null"]
items: { type: string }
missing:
type: array
items: { type: string }
required: [order, missing]
// catalog: the intent folder that holds Delivery.RankOptions.intent.yaml.
var runtime = new IntentRuntime(client, catalog);
AIResult<JsonNode> ranked = await runtime.RunAsync("Delivery.RankOptions", input);
if (ranked.Success && ranked.Value!["order"] is null)
Console.WriteLine($"Ask for: {ranked.Value["missing"]!.ToJsonString()}");
The schema is checked against every reply, so a reply that leaves out order fails its attempt and is retried, even
though order may be null (Intent file reference).
A run on scripted replies
The ranking with no delivery mode and with one, a proposal whose first answer cites a file that is not in the input, and the intent file with no delivery mode:
The ranking, with no delivery mode in the input:
Order: none
Missing: delivery mode
The same ranking, with the delivery mode "needed on site tomorrow":
Order: Courier, Pickup, Freight
A proposal from scope-letter.txt and site-visit.txt:
[Validator] attempt 1: Cite only the files in the input; ceiling-survey.pdf is not one of them.
Success: true, after 2 attempts
scope-letter.txt: Replace the 14 fixtures at night.
site-visit.txt: The ceiling is 3.2 m high, so a scaffold tower is needed.
Delivery.RankOptions, with no delivery mode in the input:
Value: {"order":null,"missing":["delivery mode"]}
The replies are scripted. A pass means the answer met the checks, not that it is right.
Limits
- IntentSharp sends what the input holds and checks what the reply holds. It cannot tell whether an input is complete for an intent: that is the contract you write.
- A value no check reads can still be wrong. Check every value a later step relies on, in a validator or a rule.
- The whole input is sent, up to
MaxInputChars. Leave out what an operation does not need, and mark a member that must never be sent with[JsonIgnore].