Break Autopilot handoff auto-fire cycles#325441
Open
Alexoswin wants to merge 3 commits into
Open
Conversation
Autopilot auto-submits a mode's send:true handoff the instant a response completes, with no user turn in between (ChatWidget#renderChatSuggestNextWidget). If a handoff chain cycles back on itself - e.g. agent A's auto-send handoff lands on agent B, whose response again offers that same handoff back to A - nothing stops Autopilot from resubmitting it forever. Each cycle is a fully completed, billed turn, and looks like a normal finished response in the UI (retry/thumbs icons and all), so there's no visible signal anything is wrong until a budget cap is hit. Track the id of the last handoff Autopilot auto-fired and refuse to fire the same one again immediately after. Legitimate multi-step chains (plan -> implement -> verify, a different handoff each time) are unaffected; only an exact repeat is suppressed, falling back to showing the handoff as a normal clickable suggestion so the user can still drive it manually. Fixes microsoft#325440
Self-review of the previous commit found two real gaps in the id-based cycle guard: 1. The guard field lived on ChatWidget but was never reset when the same widget instance got reattached to a different session (setModel / the viewModel setter). A stale "last fired handoff" from one session could spuriously suppress a brand new, unrelated session's first legitimate auto-handoff. 2. More fundamentally, comparing handoff *identities* (agent + label) across turns has a blind spot: two differently-labeled handoffs pointing at each other (e.g. "Implement" from plan -> implement, "Review" from implement -> plan) never share an id, so an A <-> B loop built that way - which is if anything the more natural way to author a back-and-forth handoff - would never be detected at all. Replaced it with a plain counter: how many send:true handoffs Autopilot has auto-fired in a row with no human turn in between, capped at AUTOPILOT_MAX_CONSECUTIVE_AUTO_HANDOFFS. This needs no assumption about handoff ids lining up across hops, so it bounds any unbounded chain - repeating or not, same ids or different - not just the specific shape the earlier check happened to catch. It resets on any non-auto-fired turn, on leaving Autopilot, and on session switch, and still falls back to showing the handoff as a manual suggestion once suppressed rather than doing nothing. Rewrote the unit tests against the new signature and added coverage for the session-switch reset and the different-id cycle shape the previous approach would have missed.
Author
|
@microsoft-github-policy-service agree |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #325440.
Autopilot auto-submits a mode's
send: truehandoff the instant a response completes, with no user turn in between (ChatWidget#renderChatSuggestNextWidget). Nothing currently bounds how many times this can happen in a row: if a handoff chain cycles back on itself — e.g. agent A's auto-send handoff lands on agent B, and B's response offers a handoff back to A — Autopilot resubmits it forever. Each lap is a fully completed, billed turn, and looks like a normal finished response in the UI (retry/thumbs icons and all), so there's no visible signal anything is wrong until a budget cap gets hit.That matches the reporter's description almost exactly: an unattended Autopilot session "restarted the thread about 40 times" overnight, each restart showing as a normal completed turn with nothing in the transcript explaining why a new one started.
Fix
Autopilot now counts how many
send: truehandoffs it has auto-fired back to back with no human turn in between, and stops auto-firing once that count reachesAUTOPILOT_MAX_CONSECUTIVE_AUTO_HANDOFFS(5 — in the same neighborhood as the existing in-turn nudge cap,ToolCallingLoop.MAX_AUTOPILOT_ITERATIONS= 3, but a little higher since a cross-turn handoff is a coarser unit of work).shouldAutoFireHandoff(consecutiveAutoFiredCount)inchatModes.tsis the pure decision function.ChatWidgettracks the count in_consecutiveAutopilotAutoHandoffs, incrementing on every auto-fire and resetting to 0 whenever: the user isn't in Autopilot mode, or the widget is reattached to a different session (session picker / history navigation go through theviewModelsetter, which is now the single reset point).Why a plain count instead of detecting the repeated handoff by id: an earlier version of this PR tried exactly that — track the id of the last auto-fired handoff and refuse to re-fire the same one. It has a real gap: two differently-labeled handoffs pointing at each other (e.g. "Implement" from plan → implement, "Review" from implement → plan) form an infinite loop whose two directions never share an id — which is if anything the more natural way to author a back-and-forth handoff, not a contrived edge case — so an identity-based check would never trip at all. It also had a session-leak bug: the guard field wasn't reset when a widget got reattached to a different session, so a stale value from one session could spuriously suppress a brand new session's legitimate first auto-handoff. A plain consecutive-attempt count has neither problem: it doesn't care whether ids match, and it's explicitly reset per-session.
This intentionally does not try to fix everything in the linked issue (the empty-response "Sorry, no response was returned" thread death and the
/compactfailure look like separate bugs in the summarization/compaction path around large-context requests) — keeping this PR scoped to the one change that stops money from being spent unattended.Test plan
chatHandoffs.test.tsforshouldAutoFireHandoff: allowed for every count under the cap, blocked at and beyond the cap, and a same-cap comparison showing a same-id cycle and a different-id cycle are governed identically (the gap the earlier id-based approach had).npm installon this monorepo is a heavyweight operation I didn't want to run unprompted). Reviewed the diff by hand against existing call sites and types, and manually traced the boundary conditions (cap reached exactly, cap exceeded, session-switch reset) against the implementation.