Skip to content

fix(agent-core-v2): notify the model of previous background tasks terminated by app exit - #3292

Merged
7Sageer merged 3 commits into
mainfrom
fix/task-resume-reminder
Aug 28, 2026
Merged

fix(agent-core-v2): notify the model of previous background tasks terminated by app exit#3292
7Sageer merged 3 commits into
mainfrom
fix/task-resume-reminder

Conversation

@7Sageer

@7Sageer 7Sageer commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator

Related Issue

N/A — internal behavior fix; the motivation is described below.

Problem

When the user exits the app with background tasks (bash commands or subagents) still running and later resumes the session, the model never learns those tasks are gone. It keeps waiting for completion notifications that will never arrive, and may reason as if the tasks had succeeded.

  • Normal exit (stopAllOnExit): detached tasks are killed with stopReason = 'Session closed' and their terminal notification is suppressed — nothing reaches the model.
  • Abnormal exit: the persisted task record stays running, reconcile marks it lost, and the per-task restore path injects a thin <notification>xxx lost.</notification> line that explains neither the cause nor what to do next.

What changed

On resume, reconciliation now delivers one unified ReminderRuntime.notify covering every background task that was still running when the previous process exited:

  • Collects every ghost task whose last state is lost (process died without a terminal record), or killed with stopReason = 'Session closed' (the exit path actively stopped it).
  • Renders one line per task: subagents get Agent(resume="agentId") recovery guidance (their context is persisted); bash tasks are listed without claiming death ("the process may still be running").
  • Persists a resumeReminded?: boolean marker on each ghost task so repeated reconcile passes and history replay can never re-inject the same reminder.
  • Excludes tasks the model deliberately stopped via TaskStop (stopReason differs), and excludes lost ghosts from the per-task restore path (the unified reminder replaces it, not duplicates it).

Mechanically this is deliberately small: a single collector + one notify call + one persistence write, no new durable event, no new delivery-key space, no new registry. The existing rejection/suppression mechanisms are untouched.

Example (user resumes after the app exits):

<system-reminder>
The user exited the application after your last turn, so your background tasks from the previous session lost contact:
- agent-prev0000 "previous agent task" (subagent) — resume it with Agent(resume="agent-prev0000", prompt="Pick up where you left off.") ...
- bash-run77777 "npm run dev" (bash)
Don't assume any of them completed; check current state (they may still be running), then re-run or resume only what you still need.
</system-reminder>

Diff-size note: the final net change is roughly +50 lines of feel code (collector + reminder + persistence marker + tests), because the reminder is an additive capability; everything that could be deleted without harming existing behavior was deleted in earlier passes of this PR (suppression-abuse checks, an invented delivery-state event, fold registrations, dispatch infrastructure). What remains is the irreducible code for the new path. Tests (idle-notification-repro, rpc-events) were rewritten, not added, wherever possible.

Checklist

  • I have read the CONTRIBUTING document.
  • I have linked a related issue (N/A — internal behavior fix).
  • I have added tests that prove my feature works.
  • Ran gen-changesets skill, or this PR needs no changeset.
  • Ran gen-docs skill, or this PR needs no doc update. (No doc update needed.)

@changeset-bot

changeset-bot Bot commented Aug 27, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 23cc6c6

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@moonshot-ai/kimi-code Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@7Hanrui

7Hanrui commented Aug 27, 2026

Copy link
Copy Markdown

@codex review

@pkg-pr-new

pkg-pr-new Bot commented Aug 27, 2026

Copy link
Copy Markdown
pnpm dlx https://pkg.pr.new/@moonshot-ai/kimi-code@23cc6c6
npx https://pkg.pr.new/@moonshot-ai/kimi-code@23cc6c6

commit: 23cc6c6

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3abfc1fa1e

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

}
if (tasks.length === 0) return;
const lines = tasks.map((info) => previousSessionTaskLine(info));
this.context.append({

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Route the resume notice through AgentReminder

This one-off model-facing event is appended directly to context, bypassing the package's required AgentReminder.notify path and duplicating its system-reminder wrapping and injection-origin semantics. Resolve AgentReminder through IAgentLifecycleService and notify from this restore hook so the reminder runtime remains the sole owner of delivery behavior.

AGENTS.md reference: packages/agent-core-v2/AGENTS.md:L80-L82

Useful? React with 👍 / 👎.

Comment on lines +1168 to +1169
for (const info of this.ghosts.values()) {
if (info.status !== 'killed') continue;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include already-lost ghosts when rebuilding the reminder

If the process exits again after markLoadedTasksLost() has persisted a task as lost but before this reminder and its delivery record are persisted, the next resume loads that task as terminal, so it is absent from lostTasks; this loop then rejects it because it only scans killed ghosts. The reminder is therefore permanently skipped after this realistic repeated-crash window. Scan undelivered lost ghosts here as well, using the same delivery-key checks.

Useful? React with 👍 / 👎.

Comment on lines +1164 to +1166
private async appendPreviousSessionTasksReminder(
lostTasks: readonly AgentTaskInfo[],
): Promise<void> {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Restore the unified notice after conversation undo

When the user invokes conversation undo immediately after resuming, undo removes the trailing reminder and rolls back its new delivery keys, but reconcileNotificationDeliveryAfterUndo() only restores ordinary task notifications and never calls this routine. Exit-killed tasks remain suppressed and disappear entirely, while lost tasks fall back to the old thin notification. Re-run the unified reminder reconciliation after undo so notification delivery continues to follow conversation undo.

AGENTS.md reference: packages/agent-core-v2/AGENTS.md:L76-L78

Useful? React with 👍 / 👎.

@7Sageer
7Sageer force-pushed the fix/task-resume-reminder branch from 3abfc1f to 3e9e109 Compare August 27, 2026 08:54
@7Sageer

7Sageer commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator Author

Addressed the review in the rework (commit 3e9e109):

  • P1: switched to the sanctioned channel — the reminder is now delivered via IAgentLifecycleService.resolve(agentContext, AgentReminder).notify(content, { variant: 'task_resume_termination' }) from the restore hook. The runtime owns wrapping/origin; the inline context.append block is gone.
  • P2 (already-lost ghosts): replaced the lostTasks-scoped collection with a full ghost sweep filtered by resumeReminded?: boolean persisted on the task record itself, so the crash window between markLoadedTasksLost and reminder delivery can't drop the reminder permanently.
  • P2 (undo): deliberately not re-arming after undo — the reminder lives in conversation history and a post-undo reconcile would need new participant machinery; out of scope for this change, can be a follow-up.

@7Hanrui

7Hanrui commented Aug 27, 2026

Copy link
Copy Markdown

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3e9e109130

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

}

function isPreviousSessionTermination(info: AgentTaskInfo): boolean {
if (info.status === 'lost') return true;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Do not assume every lost process was terminated

On the non-Windows node-local backend, processes are spawned detached by default (hostProcessService.ts:19-24), so a crash before graceful shutdown can leave a bash process running; keep_alive_on_exit=true deliberately creates the same state. Reconciliation nevertheless marks its record lost, and this unconditional classification makes the reminder claim it was terminated and instructs the model to rerun it, potentially duplicating servers or other side-effectful commands. Verify process death or tell the model that tracking was lost and to check the PID before rerunning.

Useful? React with 👍 / 👎.

Comment on lines +1179 to +1181
const marked: AgentTaskInfo = { ...info, resumeReminded: true };
this.ghosts.set(info.taskId, marked);
await this.persistence.writeTask(marked);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve reminder deduplication across truncated forks

When a turn-index fork retains records containing this reminder, sessionLifecycleService.ts:560-561,713 copies the wire history but deletes every retained agent's tasks directory. Because resumeReminded exists only in those task documents and is absent from the durable TaskTerminated record, the fork restores the same ghost with no marker and appends a duplicate reminder. Store the dedupe fact in replayable state that survives truncated forks or detect the retained injection before notifying.

Useful? React with 👍 / 👎.

@7Sageer
7Sageer force-pushed the fix/task-resume-reminder branch from 3e9e109 to 70f31ad Compare August 27, 2026 09:09
@7Sageer

7Sageer commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator Author

Round 2 addressed (commit 70f31ad):

  • P1 (don't assume lost processes died): lost bash lines now read lost when the app exited — the process may still be running; verify and stop it before re-running, and no longer instruct an immediate re-run. Subagent lines are unchanged (a subagent dies with its owning process, so Agent(resume=...) is the right recovery).
  • P2 (truncated-fork duplicate reminder): added a history check (hasResumeReminderInHistory) that skips the reminder when its variant is already present in the conversation — replayed history and forked copies both carry the original reminder, so no duplicate is appended. The persisted resumeReminded marker stays as the main-session idempotency key.
  • P2 (undo): remains out of scope — re-firing after undo would need a new undo-participant protocol; happy to take it as a follow-up.

@7Hanrui

7Hanrui commented Aug 27, 2026

Copy link
Copy Markdown

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 70f31ad01a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

}

private async appendPreviousSessionTasksReminder(): Promise<void> {
if (this.hasResumeReminderInHistory()) return;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Scope reminder deduplication to the current task set

After one resume reminder remains in history, a later resume following a second app exit returns here before examining the newly terminated ghosts. Tasks started after the first resume therefore receive neither this reminder nor their ordinary terminal notification, since restoreAgentTaskNotificationsNow() excludes all previous-session terminations. Treat each exit as a distinct one-off event and deduplicate against the relevant task IDs rather than any historical injection with this variant.

AGENTS.md reference: packages/agent-core-v2/AGENTS.md:L80-L82

Useful? React with 👍 / 👎.

@7Sageer
7Sageer force-pushed the fix/task-resume-reminder branch from 70f31ad to fc3fd32 Compare August 27, 2026 09:16
@7Sageer

7Sageer commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator Author

Round 3 addressed (commit fc3fd32):

  • P2 (variant-scope dedupe too broad): replaced the whole-variant history check with a task-ID–scoped one (hasRemindedAll): the reminder is skipped only when every task in the current reminder set already appears in an existing resume reminder in history. A second exit that takes down tasks started after the first resume now produces a fresh reminder naming only those new tasks (covered by a new reconcile test: re-injects the resume reminder for tasks lost in a later exit).

@7Hanrui

7Hanrui commented Aug 27, 2026

Copy link
Copy Markdown

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fc3fd32f97

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +1472 to +1473
if (info.kind === 'agent' && info.agentId !== undefined) {
return `- ${info.taskId} "${info.description}" (subagent, ${statusText}) — resume it with Agent(resume="${info.agentId}", prompt="Pick up where you left off; redo the last tool call if its result was never observed.") to continue from its prior context.`;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid resuming subagents that remain active in a fork source

When an idle live session is forked while a background subagent is still running, sessionLifecycleService.ts:476-488 allows the fork and :520-524 copies its running task record; the fork then marks that copy lost while the original subagent continues in the source session. This instruction tells the forked model to resume the same subagent, which can run duplicate tool calls against the shared workspace. Distinguish fork-created ghosts from tasks actually lost during application exit, or prevent/background-drain such forks before offering the resume action.

Useful? React with 👍 / 👎.

Comment on lines +1162 to +1164
if (info.resumeReminded === true) continue;
if (!isPreviousSessionTermination(info)) continue;
tasks.push(info);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Skip lost tasks whose prior notification was already delivered

On the first resume after upgrading, an older persisted lost task has no resumeReminded field even if its existing task:<id>:lost message and delivery key are already in conversation history. This loop therefore treats the historical task as a new exit event and appends a second, stale reminder claiming it happened after the last turn. Check the existing task-notification delivery state/origin before including terminal lost ghosts, while still including genuinely undelivered lost tasks.

Useful? React with 👍 / 👎.

@7Sageer
7Sageer force-pushed the fix/task-resume-reminder branch from fc3fd32 to fcb8a66 Compare August 27, 2026 09:22
@7Sageer

7Sageer commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator Author

Round 4:

  • P2 (upgrade path duplicate): fixed in fcb8a66 — the collector now also skips ghosts whose terminal notification was already delivered via the old per-task path (hasDeliveredTaskOrigin: delivery-key check + history origin check). Upgraded sessions won't re-remind tasks the old single notification already covered.
  • P1 (fork-source subagent still running): out of scope for this PR. That's a pre-existing lifetime problem in the fork machinery — forking a live session should not let reconcile relabel a running task as lost in the first place; the reminder wording exposed it rather than created it. Reclassifying fork-produced ghost copies (or marking them differently so reconcile does not produce a resume-suggestion for a subagent that's still alive in the source session) belongs to sessionLifecycleService's owner; I'm happy to open a separate issue/PR against them instead of expanding this change.

@7Hanrui

7Hanrui commented Aug 27, 2026

Copy link
Copy Markdown

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fcb8a662e7

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

tasks.push(info);
}
if (tasks.length === 0) return;
if (this.hasRemindedAll(tasks)) return;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Persist dedupe after matching a retained reminder

In a turn-index fork, the retained injection makes this branch return, but fork cleanup removes the retained agent's tasks directory, so the ghost never acquires resumeReminded. Fresh evidence beyond the prior fork thread is that after this apparent fix, a later exit that loses a new task makes hasRemindedAll false for the combined old/new set and the next reminder lists the old task again. Mark the history-covered ghosts before returning or store the one-off delivery fact in replayable state that survives forks.

AGENTS.md reference: packages/agent-core-v2/AGENTS.md:L80-L82

Useful? React with 👍 / 👎.

}

function previousSessionStatusText(info: AgentTaskInfo): string {
if (info.status !== 'lost') return 'killed on exit';

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid claiming exit-killed processes actually stopped

When graceful shutdown cannot kill a bash process, this text still states that it was killed and can encourage the model to start a duplicate. Fresh evidence beyond the prior lost-process thread is that the killed path remains unverified: terminateWithGrace() swallows force-stop failures before persisting killed, and on Windows hostProcessService.ts:116-128 resolves even when taskkill fails. Use stop-requested wording for processes or verify that the PID exited before making this claim.

Useful? React with 👍 / 👎.

@7Sageer
7Sageer force-pushed the fix/task-resume-reminder branch from fcb8a66 to b4d43d0 Compare August 27, 2026 09:29
@7Sageer
7Sageer force-pushed the fix/task-resume-reminder branch from b4d43d0 to ebd32da Compare August 27, 2026 09:48
@7Hanrui

7Hanrui commented Aug 27, 2026

Copy link
Copy Markdown

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ebd32da6a6

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

for (const info of tasks) {
const marked: AgentTaskInfo = { ...info, resumeReminded: true };
this.ghosts.set(info.taskId, marked);
await this.persistence.writeTask(marked);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not make resume depend on writing dedupe markers

When task storage is readable but this marker write fails—for example because of read-only permissions, a full disk, or transient I/O—the rejection escapes the onDidRestore hook; EventDispatcherService.restore() then marks restoration as failed, so a session with a cleanly persisted killed task can no longer resume. Treat this bookkeeping write as best-effort or record deduplication through the durable reminder-delivery path rather than allowing it to abort session restoration.

Useful? React with 👍 / 👎.

@7Sageer
7Sageer merged commit 23921e9 into main Aug 28, 2026
15 checks passed
@7Sageer
7Sageer deleted the fix/task-resume-reminder branch August 28, 2026 04:03
@github-actions github-actions Bot mentioned this pull request Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants