Skip to content

Honor store: false when recording Responses API stateful markers in BYOK#325437

Open
Davixk wants to merge 2 commits into
microsoft:mainfrom
Davixk:fix/byok-honor-store-false
Open

Honor store: false when recording Responses API stateful markers in BYOK#325437
Davixk wants to merge 2 commits into
microsoft:mainfrom
Davixk:fix/byok-honor-store-false

Conversation

@Davixk

@Davixk Davixk commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Fixes #325436

Description

BYOK custom endpoints using the Responses API record a stateful marker from every response.completed, even when the server answered store: false. The next request is then truncated to the items after the marker while a later guard strips the non-resp_ previous_response_id, producing an unanswerable request (a tool_result without its tool_use, or no user message at all).

Two changes:

  • OpenAIResponsesProcessor emits the stateful marker only when response.store !== false, so a server that declares no persistence never gets a marker recorded and later requests carry the full input. Responses without a store field keep today's behavior, so CAPI is unaffected.
  • The BYOK Responses branch validates the pending marker before the body is built: a marker not shaped resp_... sets ignoreStatefulMarker, keeping truncation and previous_response_id a single decision. The late guard remains as an invariant. This also heals existing transcripts that already carry foreign markers.

The previous_response_not_found retry flow is untouched and remains the fallback for expired server state.

How to test

  1. Configure a customendpoint provider with "apiType": "responses" against a server that echoes store: false.
  2. Run a tool-calling turn, then a follow-up in the same conversation.
  3. Before: the follow-up is truncated to post-marker items with no previous_response_id and is rejected. After: the follow-up carries the full input and succeeds.

Added spec coverage: responsesApi.spec.ts (marker emitted for store: true and absent, not for store: false) and openAIEndpoint.spec.ts (a non-resp_ marker in history yields full input and no previous_response_id).

Skip the stateful marker on response.completed when the response was
not stored, so unchainable ids never enter the history. In the BYOK
Responses branch, decide marker usability before conversion so a
non-resp_ marker sends the full input instead of a truncated one with
previous_response_id stripped by the late guard.
Copilot AI review requested due to automatic review settings July 11, 2026 15:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Prevents stateless BYOK Responses endpoints from producing truncated follow-up requests.

Changes:

  • Suppresses stateful markers when store: false.
  • Rejects non-resp_ markers before request conversion.
  • Adds regression coverage for both behaviors.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
extensions/copilot/src/platform/endpoint/node/responsesApi.ts Honors the completed response’s store flag.
extensions/copilot/src/platform/endpoint/node/test/responsesApi.spec.ts Tests marker emission across store values.
extensions/copilot/src/extension/byok/node/openAIEndpoint.ts Validates markers before input truncation.
extensions/copilot/src/extension/byok/node/test/openAIEndpoint.spec.ts Verifies foreign markers preserve full input.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BYOK custom endpoint (Responses API) sends truncated input to servers that do not store responses

5 participants