Surface SEP-2549 caching hints (ttlMs/cacheScope) on the auto-paginating list overloads

[tarekgh]

Following up on PR #1623 (SEP-2549 caching hints).

The auto-paginating client overloads (ListToolsAsync(RequestOptions?, CancellationToken), ListPromptsAsync, ListResourcesAsync, ListResourceTemplatesAsync) aggregate every page into a single IList<...> and therefore drop the per-result ttlMs / cacheScope hints introduced by SEP-2549. Today those hints are only reachable via the raw single-page overloads (e.g. ListToolsAsync(ListToolsRequestParams, CancellationToken)), which PR #1623 documents with a <remarks> note.

We should decide how (and whether) to surface the caching hints from the convenience overloads. This needs an API-design decision because:

• Each page carries its own ttlMs/cacheScope, so there is no single obviously-correct value to expose for an aggregated list (first page? minimum TTL? a per-page collection?).
• The overloads currently return IList<...>. Exposing hints would require either changing the return type (breaking), adding out parameters, or introducing a new result wrapper / additional overload.

Scope: applies to all four list overloads. ReadResourceAsync is unaffected since it is not paginated.

[tarekgh]

Resolution proposal

After reviewing SEP-2549 and the current SDK surface, the design question raised here (which TTL value to expose on the auto-paginating overloads: first page, minimum, or a per-page collection) is effectively answered by the SEP itself. The conclusion is that ttlMs is intrinsically per-page and should stay on the raw page-level overloads, the convenience overloads should remain flattened, and no aggregated TTL should be invented. Details below.

What SEP-2549 specifies about pagination

The SEP has a dedicated "Interaction with pagination" section. The relevant points:

• Each page response is an independently cacheable response. Each page carries its own ttlMs, and the freshness clock for a page starts at the time that page was received.
• Servers may return different ttlMs values on different pages (for example a longer TTL for early, stable pages and a shorter TTL for the final page).
• There is no cross-page consistency guarantee. If the underlying data changes between page fetches, clients may observe duplicates or gaps, the same trade-off as HTTP paginated APIs. A client that needs a consistent snapshot should re-fetch from the beginning without a cursor.
• If a cursor becomes invalid, the client should discard all cached pages and re-fetch from the beginning.
• cacheScope is the one hint that is uniform across pages: servers MUST apply the same cacheScope to all pages of a given list request.

Net effect: the freshness model is page-oriented. A client caches (method, cursor) entries, each with its own t_received + ttlMs expiry, and refetches only the pages that are stale. A single aggregated TTL over a flattened list cannot represent this, because it discards both the per-page TTLs and the per-page receive timestamps that the freshness math depends on.

What the SDK already supports today

All the data needed to implement the SEP's per-page caching model is already exposed. No new API is required for correctness:

• Raw page-level overloads exist for all four list methods and return the raw result:
  • ListToolsAsync(ListToolsRequestParams, CancellationToken)
  • ListPromptsAsync(ListPromptsRequestParams, CancellationToken)
  • ListResourcesAsync(ListResourcesRequestParams, CancellationToken)
  • ListResourceTemplatesAsync(ListResourceTemplatesRequestParams, CancellationToken)
• Cursor in: the request params derive from PaginatedRequest, which exposes Cursor.
• Next cursor out: the results derive from PaginatedResult, which exposes NextCursor.
• Caching hints: each result implements ICacheableResult, exposing TimeToLive (ttlMs) and CacheScope.

A cache-aware client drives pagination explicitly and has everything it needs per page (items, the cursor that produced them, ttlMs, cacheScope, and the local receive time it records itself):

var req = new ListToolsRequestParams();
do
{
    ListToolsResult page = await client.ListToolsAsync(req, ct);
    var receivedAt = DateTimeOffset.UtcNow;
    cache.Store(key: req.Cursor, page.Tools, page.TimeToLive, page.CacheScope, receivedAt);
    req.Cursor = page.NextCursor;
}
while (req.Cursor is not null);

The auto-paginating convenience overloads (ListToolsAsync(RequestOptions?, CancellationToken) and friends) intentionally flatten all pages into one IList<...> and already document, via <remarks>, that they do not surface the hints and that the raw overloads should be used to read them.

Proposed resolution

1. Do not add an aggregated ttlMs to the convenience overloads. A single reduced value (first page, minimum, and so on) is not a concept the SEP defines and would be misleading: it throws away the per-page TTLs and per-page receive times the freshness model requires, so any caller trusting it would over-cache or under-cache.
2. Keep the convenience overloads flattened and hint-free, and tighten the existing <remarks> to state plainly that cache-aware callers should use the raw page-level overloads and cache each page by cursor.
3. cacheScope is the only hint that is safely aggregatable, because the SEP guarantees it is uniform across pages. Surfacing it on the convenience layer (for example a small wrapper result) is an optional, low-risk ergonomic add, not required for correctness.

Why not ship caching helpers (a built-in page cache or caching client decorator)

This was considered and is not recommended as part of this issue:

• The SEP deliberately leaves caching as client policy, not a mechanical step. Decisions like "serve stale on refetch error?", "cheap reuse vs consistent full snapshot?", and "poll or not?" are explicitly the client's. A built-in helper would have to hard-code answers, and any fixed answer is wrong for some clients.
• The complexity is not removed by a helper, only relocated. Correct behavior still requires cursor-keyed entries, per-page expiry, the consistent-snapshot re-fetch path, the cursor-invalidation cascade, and cacheScope enforcement. Those obligations remain; a helper just centralizes them, and a flattened convenience TTL would not relieve a caller of any of them.
• cacheScope enforcement in shared or multi-tenant caches is security-sensitive (a private page must never be served to another user), which raises the support cost of anything the SDK blesses as the caching layer.
• This is a preview SDK and SEP-2549, though Final, has no reference implementation yet. Shipping an opinionated caching layer before ecosystem patterns settle risks locking in the wrong abstraction and a later breaking change.
• The audience is narrow: per-page TTL caching mainly benefits clients that both paginate and want to cache, typically in stateless deployments where push notifications are not available. Clients with small or unpaginated lists, or that simply refetch on demand, do not need it. list_changed is not a substitute here, since the stateless deployments that most need TTL caching are exactly the ones that cannot rely on push notifications.

If there is appetite for an optional, unopinionated page-cache utility (caller still drives the loop, policy stays with the caller), I suggest tracking it as a separate exploratory issue so it can be designed against real-world usage rather than speculatively.

Suggested closing actions for this issue

• Confirm the decision: TTL stays page-level on the raw overloads; convenience overloads stay flattened.
• Optionally tighten the convenience-overload <remarks> wording.
• Optionally surface the uniform cacheScope on the convenience layer.
• Optionally open a follow-up issue for an opt-in page-cache helper.