Add a routing context factory for request-scoped policy state

[joshuajyue]

Follow-up to #7662.

Summary

Adds one protected virtual method to RoutingChatClient:

protected virtual RoutingContext CreateContext(
    IEnumerable<ChatMessage> messages,
    ChatOptions? options) => new(messages, options);

GetResponseAsync and GetStreamingResponseAsync call it instead of constructing a RoutingContext directly, so a derived class can return its own subclass and have it flow through selection and, for FailoverChatClient, every routing update for that request.

OrderedFailoverChatClient moves onto it, which removes its ConcurrentDictionary<RoutingContext, int>.

Why

Nothing here is newly possible. A router can already keep whatever it wants in a side table keyed by the context — that is what OrderedFailoverChatClient does today for its next-client index, and RoutingContext does not override Equals/GetHashCode, so reference-keying is sound. The question is only where that state should live.

	side table	context state
Capability	same	same
Lifetime	manual: every exit path must remove the entry	automatic: collected with the context
Leak if a request ends without a terminal update	yes	not possible
Concurrency	needs ConcurrentDictionary; every access hashes and may contend	a field on a per-request object; no synchronization
Access cost	hash lookup per selection and per update	field read
Typing	dictionary value type, unpacked at each use	typed property
Disposal	must clear the dictionary	nothing to clear

The leak is the substantive row. State must survive from a nonterminal update to the following selection, so it cannot be removed eagerly, and a request can end without a terminal update — an abandoned streaming enumerator, or selection failing after state was already stored. Today the derived class is responsible for cleaning up on those paths. Context-owned state hands that responsibility to the GC.

The rest is ergonomics, and it compounds when a selector needs more inputs rather than more state. SelectClientAsync takes a RoutingContext, so a router wanting to hand its callback anything else has to carry a second object and correlate the two. Agent Framework's routing client (microsoft/agent-framework#6932) needs the current agent, the session, its registered destination map, and the session's active destination available to its routing callback, so it declares a standalone context type and copies messages and options into it. That works, and it is why Microsoft.Agents.AI.RoutingContext and Microsoft.Extensions.AI.RoutingContext are currently unrelated types with the same name: two objects per request, and a routing callback written against one does not compose with the other.

RoutingContext is a non-sealed public class with a public constructor, so it is already shaped for that subclass to exist. The base class just never lets one through — both invocation methods hardcode new RoutingContext(...), so whatever a derived class builds cannot reach SelectClientAsync. This finishes that rather than introducing a new concept.

Two things worth noting: this is not ambient state, since the context is an explicit parameter to SelectClientAsync and OnRoutingUpdateAsync and nothing depends on ExecutionContext flowing — which is what the AsyncLocal suggestion in #7662 was aimed at, without the SuppressFlow hazard raised against it there. And RoutingContext still clones the caller's options in its constructor, which a subclass has to call, so the caller's instance stays protected either way.

Trade-offs

• The downcast is unchecked. A dictionary is type-safe by construction; a context subclass is reached by casting. That is sound when the router is sealed, as OrderedFailoverChatClient is, and weaker when it is not.
• One context per request, of one type. Two independent layers cannot each attach state unless one's context derives from the other's. A dictionary has no such constraint. This does not bite today because FailoverChatClient stores nothing on the context, but it does mean that if the base class ever wants its own context state, existing derived contexts would have to re-parent.
• New failure mode. CreateContext can return null, so both invocation methods guard for it.

Changes

• RoutingChatClient — adds CreateContext; both invocation methods call it, with a null guard matching the existing SelectClientAsync check.
• FailoverChatClient — no new API. It inherits CreateContext, and because it declares both invocation methods sealed override, the factory is guaranteed to run for its derived types.
• OrderedFailoverChatClient — moves its next-client index onto a private context subclass; removes the ConcurrentDictionary, the per-call lookups, both cleanup paths, and the _requestStates.Clear() in Dispose.
• Tests — CreateContext coverage at both the RoutingChatClient and FailoverChatClient levels, including a null return; OrderedFailover_AbandonedStreamDoesNotAffectLaterRequests replaces the test that used reflection to inspect the removed dictionary.

Notes

• A derived class that overrides CreateContext casts the context back to its own type in SelectClientAsync. OrderedFailoverChatClient asserts the type rather than checking it at runtime: it is sealed, FailoverChatClient seals both invocation methods, and selection and updates are protected, so the only context those methods can receive is the one CreateContext produced. Reaching a different type would take a change to the library itself, which the assert catches in Debug.
• Additive and experimental: the virtual has a working default, so existing subclasses are unaffected.
• A generic RoutingChatClient<TContext> would remove the downcast, at the cost of a type parameter across the whole hierarchy.

Validation

• Build clean on all target frameworks, 0 warnings.
• Microsoft.Extensions.AI.Tests — 762 passed.
• Microsoft.Extensions.AI.Abstractions.Tests — 1646 passed.
• API manifests updated by hand and verified against MakeApiBaselines.ps1 output.

[jozkee]

Nice, thanks.

[Copilot]

Pull request overview

Adds an extensibility point to chat routing so derived routing clients can create a per-request RoutingContext (including custom subclasses) and have it flow consistently through selection and (for failover) routing updates, enabling request-scoped policy state without side tables.

Changes:

• Add protected virtual RoutingContext CreateContext(IEnumerable<ChatMessage>, ChatOptions?) to RoutingChatClient and use it in both non-streaming and streaming entry points (with a null-return guard).
• Update FailoverChatClient to use CreateContext in its sealed overrides and document the request-scoped-state pattern.
• Refactor OrderedFailoverChatClient to store its “next client index” on a private RoutingContext subclass, removing the per-request ConcurrentDictionary and related cleanup paths; update tests and API manifests accordingly.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
test/Libraries/Microsoft.Extensions.AI.Tests/ChatRouting/OrderedFailoverChatClientTests.cs	Replaces reflection-based state-leak test with an abandoned-stream scenario to validate state scoping.
test/Libraries/Microsoft.Extensions.AI.Tests/ChatRouting/FailoverChatClientTests.cs	Adds coverage ensuring custom contexts flow through selection and updates; adds null-context guard coverage.
test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/ChatRouting/RoutingChatClientTests.cs	Adds coverage ensuring custom contexts flow to selection; adds null-context guard coverage.
src/Libraries/Microsoft.Extensions.AI/Microsoft.Extensions.AI.json	Updates API manifest to include OrderedFailoverChatClient.CreateContext override.
src/Libraries/Microsoft.Extensions.AI/ChatRouting/OrderedFailoverChatClient.cs	Moves ordered failover request state onto a context subclass; removes side-table state.
src/Libraries/Microsoft.Extensions.AI/ChatRouting/FailoverChatClient.cs	Switches to CreateContext for request context creation and documents request-scoped context state.
src/Libraries/Microsoft.Extensions.AI.Abstractions/Microsoft.Extensions.AI.Abstractions.json	Updates API manifest to include RoutingChatClient.CreateContext.
src/Libraries/Microsoft.Extensions.AI.Abstractions/ChatRouting/RoutingChatClient.cs	Introduces CreateContext factory and uses it in both invocation paths with a null guard.

[joshuajyue]

Closing this for the same reason the RoutingSelection proposal was dropped: it doesn't enable anything new, and the experimental surface should stay small.

Anything a derived context could carry, a side table keyed by the context can carry too — that's what OrderedFailoverChatClient does today. CreateContext makes it tidier, one object instead of two with no cleanup path to get wrong, but tidier isn't enough to justify permanent API on a surface that just shipped, for now.

It's additive and purely a virtual with a working default, so it can land later without breaking anyone. If a concrete consumer hits a wall, that's a better time to add it, with the actual requirements in hand.

The trade-offs support waiting too: the downcast isn't type-checked, there's one context per request of one type so two layers can't independently attach state, and if FailoverChatClient ever wants its own context state, existing derived contexts would have to re-parent.

Thanks @jozkee for the review.