Skip to content

Thinking router re-fires on every timeout/auth resume slice #565

Description

@sentry-junior

selectTurnThinkingLevel makes a Haiku LLM call to pick a complexity level for the upcoming turn. It is called unconditionally every time generateAssistantReply runs — including on every timeout-resume and auth-resume slice, where the turn context is identical to the original invocation.

Root cause: respond.ts:843 has no resumedFromSessionRecord guard, and AgentTurnSessionRecord does not persist thinkingSelection, so resumed slices can't skip or reuse the prior result.

Impact: each resume slice pays an extra Haiku round-trip for a classification that will return the same answer (temperature: 0, same input). With AGENT_TURN_TIMEOUT_RESUME_MAX_SLICES = 48, a long sandbox task can generate 10–48× the expected router calls per turn.

Affected files:

  • packages/junior/src/chat/respond.ts — unconditional selectTurnThinkingLevel call at line 843
  • packages/junior/src/chat/state/turn-session.tsAgentTurnSessionRecord has no thinkingSelection field
  • packages/junior/src/chat/services/turn-session-record.tspersistRunningSessionRecord / persistAuthPauseSessionRecord / persistTimeoutSessionRecord need to carry the selection forward

Proposed fix:

  • Add an optional thinkingSelection field to AgentTurnSessionRecord and its storage/parse path (backward-compatible, fall back to recompute if absent)
  • Persist the selection at the same time the first safe Pi boundary is written
  • Skip selectTurnThinkingLevel on resume when a persisted selection is available, adding a "app.ai.thinking_level_source": "session_record" | "router" span attr for traceability

See also: #431 (compaction rework, related session-record work)

Action taken on behalf of David Cramer.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions