Models per intent
The IntentSharp.Specialist package lets developers and programs define models per intent, so an application can use
several models easily, within one context or across several. One intent can run on a small model made for it,
another on a different one, and the rest on the client's usual providers, through one IntentClient and one catalog.
For each catalog intent, a SpecialistRuntime obtains and uses a model from a specialist service you wire in. Every
answer is checked against the intent's shape, rules and validators, as with any other provider.
The package never trains, tunes or changes a model, and it names no specialist service or provider: the adapter to
your service is your code, an ISpecialistService. It depends on exactly its own version of IntentSharp and
IntentSharp.Catalog, and your IntentSharp license covers it like every other IntentSharp package.
A service of your own, with the providers you have
The package is useful with no outside service at all. A small ISpecialistService of your own can hand back, for
each task, a version of the intent tuned for it (its wording, examples and output requirements) with the name of a
provider and model your client already has. In effect it attaches a tuned intent and a chosen model to each task, by
name:
/// <summary>
/// A specialist service of your own. For one task it hands back a version of the intent tuned for that task, to run on
/// a provider and model the client already has, chosen by name. Any other intent gets no offer.
/// </summary>
public sealed class TunedIntentService : ISpecialistService
{
private readonly System.Collections.Concurrent.ConcurrentDictionary<string, SpecialistRequest> _requests = new();
public string Name => "tuned-intents";
public Task<SpecialistTicket> RequestAsync(SpecialistRequest request, CancellationToken cancellationToken)
{
var ticket = new SpecialistTicket { Id = Guid.NewGuid().ToString("N"), ServiceName = Name, RequestHash = request.RequestHash, RequestedAt = DateTimeOffset.UtcNow };
_requests[ticket.Id] = request;
return Task.FromResult(ticket);
}
public Task<SpecialistOffer?> WaitAsync(SpecialistTicket ticket, TimeSpan timeout, CancellationToken cancellationToken)
{
var request = _requests[ticket.Id];
if (request.IntentName != "Orders.ClassifyUrgency") return Task.FromResult<SpecialistOffer?>(null);
// A copy of the requested intent, with the same name, shape and rules; its wording and examples are tuned.
var tuned = IntentFile.Read(IntentFile.Write(request.Definition), "tuned.intent.json").Definitions[0];
tuned.Intent = "Rate how urgent this order email is for the warehouse, from 1 (routine) to 5 (a line stops).";
tuned.OutputRequirements = "Answer 5 only when production or a delivery stops.";
tuned.Examples.Add(new IntentExample("Please add two boxes of labels to next month's order.", "1"));
return Task.FromResult<SpecialistOffer?>(new SpecialistOffer
{
Definition = tuned,
ProviderName = "small", // a provider the client already has
Model = "small-model", // the model to ask it for
Info = new SpecialistInfo { Name = "urgency-tuned", RequestHash = request.RequestHash }
});
}
public Task ReportFitAsync(SpecialistTicket ticket, FitReport report, CancellationToken cancellationToken) => Task.CompletedTask;
public Task RetireAsync(SpecialistTicket ticket, CancellationToken cancellationToken) => Task.CompletedTask;
}
// "small" stands for a provider your client already has, from its provider file or added in code.
var usualModel = new ScriptedProvider { Name = "main" };
var smallModel = new ScriptedProvider("{\"value\":5}") { Name = "small" };
using var client = new IntentClientBuilder()
.UseProvider(usualModel)
.AddProvider(smallModel) // reachable by name only
.Build();
var specialists = new SpecialistRuntime(client, catalog, new TunedIntentService());
AIResult<int> result = await specialists.RunAsync<int>("Orders.ClassifyUrgency",
"Line 3 is down until the replacement belts arrive.");
SpecialistStatus status = specialists.GetStatus("Orders.ClassifyUrgency"); // Active, on "urgency-tuned"
The runtime checks that the returned intent still fits (the same name and result shape, and no rule dropped or
changed), keeps the specialist's name and the returned intent, reuses them on later calls, and asks again when the
requested intent changes. The run answers on small with the tuned intent; the usual provider is not asked. When
the service is later replaced by one that provides a model made for the task, the code that runs the intent does not
change: only the service you pass to the runtime does.
The exchange
- Request. The runtime sends the service the intent's current definition, the context your context sources return and your requirements, and receives a ticket. The same request is never sent twice.
- Wait. It asks the service for an offer until one arrives or the waits run out, before the call starts. When
every wait ends empty the call throws
SpecialistWaitTimeoutException; the request stays open, and the next call waits on it without sending another. - Offer. An offer carries the definition to run and one way to reach the model:
ProviderName, a provider the client already has, orProvider, one your adapter built, reached through aSpecialistRouter. - Fit. The fit evaluators judge the offer, and the service receives the verdict. By default the offered
definition must keep the requested name and shape, every requested rule, validator name and guardrail set, and,
when the offer says so, answer this request. An offer that does not fit is never run: the call throws
SpecialistNotFitException, unless yourINotFitHandlerrequests another definition. - Use. The intent runs on the specialist with the definition the service returned, under its own name.
- Change. A call that finds a specialist in use for the same definition runs at once. A changed definition
requests again on the next call, and callers wait for the new specialist unless
KeepPreviouslets the one in use serve while the change is judged.
An offer that brings its own provider
When the service builds a provider of its own for the specialist, the client reaches it through a
SpecialistRouter. This example runs offline on the scripted service and scripted models the package ships for
tests:
// Scripted stand-ins: the application's usual model, the model the service offers, and the service itself.
// The offered model's first reply, 9, breaks the intent's range; its second, 4, passes.
var usualModel = new ScriptedProvider { Name = "usual" };
var urgencyModel = new ScriptedProvider("{\"value\":9}", "{\"value\":4}") { Name = "urgency-model" };
var service = new ScriptedSpecialistService(
ScriptedSpecialistService.ReturnRequested(urgencyModel, specialistName: "urgency-small"));
// The router reaches the model an offer brings. The client knows it by name only.
var router = new SpecialistRouter();
using var client = new IntentClientBuilder()
.UseProvider(usualModel)
.AddProvider(router)
.AddSpecialistGuardrails() // declares the four specialist settings and the six subjects
.Configure(options => { options.MaxAttempts = 2; options.MaxModelCalls = 2; })
.Build();
var catalog = new InMemoryIntentCatalog([new IntentDefinition
{
Name = "Orders.ClassifyUrgency",
Version = 1,
Intent = "Rate the urgency of this order email from 1 (routine) to 5 (critical).",
Shape = new IntentShape { Kind = IntentShapeKind.Primitive, Primitive = "int" },
Rules = new IntentRules { Minimum = 1, Maximum = 5 }
}]);
// Requests, verdicts and held specialists are kept in a document store (here one in memory).
var state = new InMemoryDocumentStore();
var specialists = new SpecialistRuntime(client, catalog, service, new SpecialistOptions(), router,
store: new DocumentSpecialistStore(state));
// Requests a specialist, waits for the offer, judges it, then runs the intent on it with the usual checks.
AIResult<int> result = await specialists.RunAsync<int>("Orders.ClassifyUrgency",
"Our packaging line stops on Thursday unless the replacement belts arrive.");
The run sends one request, puts the offered specialist in use, and answers on it: the range rule refuses the first reply and the retry passes with 4. The usual model receives no call. This shows the exchange and the checks with fixed replies; it measures no model.
A provider added with AddProvider is reachable by name only: it never becomes the primary provider or part of a chain
the client makes up itself. A Request checkpoint rule on request.host sees the host each routed call reaches.
RunAsync (any shape, as JSON) and RunManyAsync (one call per input, results in input order) work the same way.
With WaitOn set to Explicit, only EnsureAsync requests and waits, and a run with no specialist in use throws
SpecialistNotEnsuredException:
// Runs never wait: only EnsureAsync requests and waits, here once at startup.
var options = new SpecialistOptions { WaitOn = SpecialistWaitMode.Explicit };
var specialists = new SpecialistRuntime(client, catalog, service, options, router);
try
{
SpecialistStatus status = await specialists.EnsureAsync("Orders.ClassifyUrgency");
Console.WriteLine($"{status.IntentName}: {status.State}, specialist {status.SpecialistName ?? "(no name)"}");
}
catch (SpecialistWaitTimeoutException)
{
// Every wait ended empty. The request stays open, and the next ensure waits on it without sending another.
}
catch (SpecialistNotFitException notFit)
{
foreach (var problem in notFit.Report.Problems()) Console.WriteLine(problem);
}
RefreshAsync gathers the context again and requests when it changed; RetireAsync tells the service, stops using
the specialist and forgets what the store holds for it. Under dependency injection:
services.AddIntentSharp(configuration);
services.AddIntentCatalog("intents");
services.AddSingleton(service); // your adapter to your specialist service
services.AddIntentSharpSpecialist(o =>
{
o.AddRouter = true; // offers that bring a provider are reached by name only
o.AddGuardrailDeclarations = true; // rule files and the settings document know the specialist settings
});
Options and their defaults
var options = new SpecialistOptions
{
WaitTimeout = TimeSpan.FromMinutes(2),
MaxRequestsPerHour = 2,
// The context a request carries: the application's other intents (names, descriptions and tags).
ContextSources = { new CatalogContextSource(catalog) },
// Added to the definition comparison, which stays: the offered intent must answer this case with 4.
FitEvaluators =
{
new ProbeCaseEvaluator([new ProbeCase
{
Name = "belts",
Input = "Our packaging line stops on Thursday unless the replacement belts arrive.",
Expected = JsonValue.Create(4)
}])
},
// One intent keeps its specialist in use while a changed definition is requested and judged.
Intents =
{
["Orders.ClassifyUrgency"] = new SpecialistIntentOptions { KeepPrevious = KeepPreviousPolicy.Always }
}
};
// state: a document store that keeps requests, verdicts and held specialists across restarts and processes.
var specialists = new SpecialistRuntime(client, catalog, service, options, router,
store: new DocumentSpecialistStore(state));
| Option | Default | What it decides |
|---|---|---|
WaitOn |
IntentCall |
When a specialist is requested and waited for: on the intent call, or only in EnsureAsync (Explicit). |
WaitTimeout |
30 seconds | The length of one wait. |
WaitPollInterval |
1 second | The pause before asking the service again within one wait. |
WaitRetries |
3 | The further waits after the first. |
MaxRequestsPerHour |
3 | The most requests for one intent in any hour; one more throws SpecialistRequestLimitException and is not sent. |
ContextSources |
none | What a request carries besides the definition and the requirements. |
Requirements |
none | What the specialist must meet, in a form your service reads. |
FitEvaluators |
the definition comparison, in code | The checks every offer goes through; ProbeCaseEvaluator runs cases on the offered specialist. |
AIJudge |
off | A model you name asks whether the offer meets the request. |
NotFitHandler |
none: the call throws | After an offer does not fit, end or request another definition. |
KeepPrevious |
Never |
Whether the specialist in use serves during a change. |
HoldSpecialists |
true |
Whether a held record is kept when a specialist is put in use. |
UseHeld |
Use |
Whether an ensure puts a fitting held specialist in use from the store. |
GuardrailSets |
none | The rule sets checked at the specialist checkpoint before each request. |
CharactersPerToken |
3 | The characters counted as one token for the specialist's input limit. |
Intents overrides any of them for one intent. The store is in memory by default; DocumentSpecialistStore keeps
requests, verdicts and held specialists in a document store, so several processes share one request per intent and a
restarted process resumes the same wait. A settings document or a rule set's settings: block can set the four wait
and request limits, and a file's value beats the code value
(Settings reference).
Keeping a specialist's name and intent
When a specialist is put in use, the runtime keeps a held record: the specialist's name exactly as the service gave
it (none when the service gave none), the returned definition as intent-file text, how the model is reached and the
verdict. Every run is tagged specialist.name.
// Where the intent stands, and the held record: the specialist's name as the service gave it and the returned intent.
SpecialistStatus status = specialists.GetStatus("Orders.ClassifyUrgency");
HeldSpecialist? held = await specialists.GetHeldAsync("Orders.ClassifyUrgency");
HeldSpecialist? byName = await specialists.FindHeldAsync("urgency-small");
IntentDefinition? returned = held?.ReadDefinition();
string? intentFileText = held?.DefinitionText;
// After a restart, a new runtime over the same document store puts the held specialist in use again, with no
// new request and no wait. An offer that brought its own provider is recalled from the service.
var later = new SpecialistRuntime(client, catalog, service, new SpecialistOptions(), router,
store: new DocumentSpecialistStore(state));
SpecialistStatus again = await later.EnsureAsync("Orders.ClassifyUrgency");
bool fromHeld = again.FromHeld;
ListHeldAsync lists every held specialist, and FindHeldAsync compares names exactly. A held record made for the
same intent and request, whose verdict fits, is used again without a new request.
What leaves your application
A request sends the service the intent's definition, as the catalog holds it, the context items your sources return
and your requirements: nothing a source did not return. Fit reports go back to the service. A rule set at the
specialist checkpoint can refuse a request before anything is sent, and the call throws
SpecialistGuardrailException. Runs on the specialist, probe cases and the AI judge are ordinary calls through the
client, with its checkpoints and limits; the specialist's provider receives what any provider receives
(Security).
What it does not do
- It does not train, tune or change a model.
- It names no specialist service or provider and ships no wire format: the adapter is yours.
- It does not wait for or notify a person: a not-fit handler decides in code.
- It does not run an offer that did not fit, and it does not fall back to another provider when the waits run out.
- A passing result means the configured checks passed, not that the answer is right.