Results and verification
Every call returns an AIResult<T>. It says whether an answer passed every check, holds the checked value, and
records each problem and each attempt. There is no confidence score: a result reports only what was checked. This
page covers what the result holds, the checks every answer gets, and the two ways to add checks of your own, rules
in code and a judge.
Success means every check passed
Read Success before Value. When it is true, Value holds an answer that passed every check below. When it is
false, Value is the type's default and Issues says what went wrong:
if (order.Success)
Console.WriteLine($"{order.Value!.Quantity} x {order.Value.Sku} for {order.Value.Contact}");
else
foreach (var issue in order.Issues)
Console.WriteLine(issue); // where it was found, on which attempt, and what is wrong
A failed call does not throw. A refusal, an answer that fails a check, an error from a provider and a limit that was
hit are all issues on a failed result. How a call works lists the few mistakes that throw
before a call starts. The database methods are the exception: they return a QueryResult, and throw
SemanticDatabaseException when no safe statement results (see Database queries).
What a result holds
| Member | What it holds |
|---|---|
Success |
True when an attempt's answer passed every check. |
Value |
The checked value on success; the type's default otherwise. |
Issues |
Every problem found, on every attempt, as a ResultIssue: where it was found, the attempt, and the message. |
Attempts |
One AttemptRecord per attempt that finished its checks, in order: the provider and the model that answered, the tokens, the time, whether it passed, the reply text (RawOutput) and that attempt's issues. |
Provider, Model |
The provider that gave the final reply, and the model its reply named. |
Usage |
Tokens for the whole call, judge calls included. |
Elapsed |
Time for the whole call, every attempt included. |
RawOutput |
The final reply text, before it was read into your type. |
FromCache |
True when the answer came from the request cache. |
Intent |
The intent the call ran. |
Warnings |
Advice about the intent text. It never changes Success. |
FailureSummary |
The distinct issue messages on one line, or null on success. |
Two methods read the value for you. GetValueOrThrow() returns it, or throws an IntentSharpException whose message
lists the distinct issue messages. GetValueOrDefault(fallback) returns it on success and your fallback otherwise.
The result holds every reply, but not the requests that produced them. To see those, log them as shown in Seeing every request.
Issues say where, when and what
Each ResultIssue has a Source, the Attempt it belongs to (counted from 1), and a Message written so that a
model could act on it.
Source |
The problem |
|---|---|
Provider |
The model refused, the reply was cut off by the output token limit, or the provider failed: an HTTP error, a timeout or a transport error. |
Verification |
The reply was empty, was not JSON in the expected form, did not fit your type, missed a required member, was null where a value is required, or named an enum value your enum does not have. |
Validator |
One of your rules or validators, or a check a collection method or the database runtime adds, rejected the value. |
Judge |
The judge listed a problem. |
Guardrail |
The call reached its model-call ceiling or its time budget. |
An issue raised when a limit or a provider error ends the call carries the number of the attempt it stopped, which
is not in Attempts. When that attempt already had a reply, the reply's tokens are not in the result's Usage,
though the client's totals count them.
The checks every answer gets
Every answer is checked in your process, with no model call, in this order:
- Not refused. A refusal is an issue, and the call ends with no further attempt.
- Not cut off. A reply cut off by the output token limit fails its attempt.
- Not empty.
- It fits your type. For a text result, the reply text is the value. For any other result, the reply must be JSON in the expected form that reads into your type: every required member present, no null unless the type allows one, and enum values given by name, as a member of your enum. Typed output says which members are required.
- Your rules pass. Your rules and validators run on the typed value, then any checks the method you called adds for itself.
A failed check becomes an issue, and its message goes back to the model on the next attempt.
What a pass does not prove
A pass means these checks passed; checks narrow the answers, and they cannot prove an answer right. An answer that fits your type and breaks no rule can still be wrong, such as the wrong SKU from an email that names two. Ranges, lengths and patterns are not part of the schema the model receives, so check them in your rules. For a shape chosen at run time as a JSON schema, the checks above confirm only that the reply is JSON and not null; your validators, or an intent file's schema shape, check the rest. Where a check in code cannot decide, add the judge.
Add your own checks in code
Require<T> takes a condition and the message to report when it is false. Validate<T> returns any number of
messages, and none means the value is acceptable. Both go on the request options:
// Your own checks on the typed value. A failed check is an issue, and the next attempt sends it to the model.
var options = new AIRequestOptions()
.Require<PurchaseRequest>(r => r.Quantity > 0, "Quantity must be a positive number.")
.Validate<PurchaseRequest>(r => r.DeliverBy < DateOnly.FromDateTime(DateTime.Today)
? [$"DeliverBy is {r.DeliverBy:yyyy-MM-dd}, which is in the past."]
: []);
var order = await email.AIAsync<PurchaseRequest>(client, "Extract the purchase request from this email.", options);
Your checks run only on an answer that read into your type, and they run on every attempt. A check that throws does not throw out of the call: its message becomes a finding, and that attempt fails like any other. The model reads every message on the next attempt, so write each one as an instruction ("Quantity must be a positive number.") and keep secrets and personal data out of it.
The same checks can come from other places:
- On an intent method, an
[AIGuardrail]of the kindGuardrailType.CustomValidator, given your validator's type, runs a class that implementsIGuardrailValidatorand has a constructor with no parameters. - In an intent file, rules such as
minimum,patternorenumare checked with no code, and the file can name C# validators your application registers. See Intent files. - The collection methods add a check of their own: a sort must return every element exactly once, for example. See Collections.
The judge: a second model reviews the answer
The judge is an optional model call that reviews an answer after it passes the checks above. It reads the intent, the
input and the answer, and either accepts the answer or lists its problems. Each problem is a Judge issue, and the
next attempt sends it back like any other finding. Here the first answer fits the type but has the SKU wrong:
// JudgeMode.OnRequest is the default: a judge runs only for a request that sets UseJudge.
using var client = new IntentClientBuilder()
.UseProvider(provider)
.Configure(o => o.JudgeMode = JudgeMode.OnRequest)
.Build();
var order = await email.AIAsync<PurchaseRequest>(client, "Extract the purchase request from this email.",
new AIRequestOptions { UseJudge = true });
Scripted model answer (attempt 1): {"value": {"contact": "Dana (Northwind)", "sku": "A-1129", "quantity": 40, "deliverBy": "2026-11-15"}}
Scripted model answer (judge, attempt 1): {"passes": false, "problems": ["The email asks for SKU A-1192, not A-1129."]}
Scripted model answer (attempt 2): {"value": {"contact": "Dana (Northwind)", "sku": "A-1192", "quantity": 40, "deliverBy": "2026-11-15"}}
Scripted model answer (judge, attempt 2): {"passes": true, "problems": []}
Success: true
Value: PurchaseRequest { Contact = Dana (Northwind), Sku = A-1192, Quantity = 40, DeliverBy = 2026-11-15 }
Issues:
[Judge] attempt 1: The email asks for SKU A-1192, not A-1129.
Attempts: 2, model calls: 4 (2 answers and 2 judge calls)
The answers and verdicts in this output come from a scripted model, not a real one.
When the judge runs is set once, on the client, with IntentClientOptions.JudgeMode:
JudgeMode |
The judge runs |
|---|---|
Never |
Never. No judge runs, even one given with WithJudge. |
OnRequest (the default) |
For a request that sets UseJudge on its options, or an intent method with an [AIGuardrail] of the kind GuardrailType.RequireJudge set to true. |
Always |
On every attempt that passes the checks. |
The default judge asks the primary provider, with JudgeModel when you set one and the provider's default model
otherwise. To judge some other way, pass your own IResultJudge to WithJudge. What is sent
lists what a judge call carries.
A judge call costs one model call per attempt it reviews. It counts toward MaxModelCalls, and its tokens are in
the result's Usage and the client's totals, but not in any AttemptRecord.
Limits of the judge:
- It is a model, so it can be wrong too. Rules in code are cheaper and exact wherever they can decide.
- If a judge reply is not JSON, the attempt passes and a
Warningis logged, so a broken judge never blocks your results. Nothing on the result marks this; watch the log atWarning. A JSON reply without a passing verdict fails the attempt.
Next
- Typed output: which types get a strict schema, and the options that shape an answer.
- How a call works: attempts, limits, the cache and the fallback.