fix(codex): finalize Codex sessions instead of leaving them to pile up

[Soph]

https://entire.io/gh/entireio/cli/trails/1001

Closes the user report that Codex sessions pile up and have to be cleaned by hand with entire status + entire session stop <id>.

The request was for entire session stop --older-than=3d. That treats the symptom — the root cause is fixable at the source, and process death is a better signal than any time threshold. No new flag is included; see "On --older-than" below.

What was wrong

1. Codex never fired a session-end hook. We registered SessionStart, UserPromptSubmit, Stop and PostToolUse. Codex gained a SessionEnd hook in 0.146 (openai/codex#33895); we weren't listening, so quitting Codex left the session un-finalized forever.

2. The safety net had an off-by-one-phase bug. State.OwnerExited() gated on Phase.IsActive(), which is only PhaseActive. An agent that finishes its last turn transitions ACTIVE → IDLE, so a session whose agent then quit was invisible to the dead-owner sweep in entire status and entire doctor. That is precisely the shape Codex leaves behind — and it can happen to any agent killed before its hook runs.

Those sessions lingered as "active" for 7 days until StaleSessionThreshold made StateStore.Load delete the state file outright, discarding pending checkpoint work rather than condensing it.

Verified against real Codex

Same flow run twice, with a binary built at main for the baseline:

	before	after
session state	phase: idle, ended_at: null	phase: ended
entire status	lists it under Active Sessions	nothing lingering

The session-end hook completes in 19ms against Codex's 3s cap. The safety net was exercised independently by pointing the new binary at the stale pre-fix repo: Finalized 1 exited session(s) (agent process gone), idle → ended. That path covers Codex < 0.146, SIGKILL, and reboot.

Notable constraint

SessionEnd runs inside Codex's shutdown, so it is budgeted far more tightly than any other hook: 1s default, hard 3s cap (SESSION_END_MAX_TIMEOUT_SEC), and on expiry Codex terminates the hook's entire process tree. Asking for more prints a clamping warning at every startup.

So hook timeouts became per-event, and the command match now includes the timeout so a hook installed by an older Entire gets rewritten rather than left to nag. Agents in this position declare agent.SessionEndBudgeter, which bounds only the eager condense — mark-ended is a single atomic rename and always runs to completion, so a session can never be left un-finalized.

Also fixed here

• A stall this PR would otherwise have introduced. Widening the sweep to IDLE multiplied its candidate set, and each candidate triggered a deadline-free condense inside entire status — ~120ms floor, seconds for a real transcript, serial and uncapped, in a 40ms command. --json is what the MCP entire_status tool calls, and prints nothing until it returns. sweepCondenseBudget now caps condensing across the whole sweep while still marking every candidate ENDED.
• An internal log line leaking into entire status. Neither status nor doctor initializes logging, so the sweep's phase transition line printed raw onto the terminal via slog.Default(). Pre-existing, but this PR makes it common. Now routed to .entire/logs/ — and the level getter is wired, without which the sweep ignored log_level in settings.

On --older-than

Deliberately not included. Once process death handles this automatically, a time threshold only serves cases where liveness is Unknown: Windows (proclive is unsupported there) and cross-host state. Happy to add it if you still want the escape hatch.

Needs a follow-up decision

codex exec now fires hooks — all five, verified against 0.147.0 — which invalidates a premise entire review is built on. With ENTIRE_REVIEW_* set, the session is tagged kind: agent_review with skills and prompt captured.

But an end-to-end entire review in a fresh repo still produces zero hooks and no session, and the gate is hook trust, not the sandbox. Holding review's argv fixed and varying one flag: --dangerously-bypass-hook-trust → 5 hooks and a tagged session; -s workspace-write alone → nothing. Codex silently skips hooks with no trusted_hash, and codex exec is non-interactive so it can never prompt for it. (-s read-only suppresses hooks even when trust is bypassed, so the sandbox is a second, independent gate.) The run.Buffer fallback stays required. Documented and flagged rather than acted on — reworking review needs a trust-bootstrapping decision first and doesn't belong in a session-cleanup fix.

Review follow-ups

Four bot comments, all fixed in 1d36ecb67, plus a hole the first one led to.

AreHooksInstalled was all-or-nothing (Bugbot). Adding SessionEnd to the required set un-installed Codex for everyone who enabled it before this release. Worse than reported: DetectPresence delegates to it, so such a repo was neither installed nor detected — Codex disappeared from entire status, the review and investigate pickers, and a non-interactive entire enable re-run (which keeps the currently installed agents) would have dropped Codex rather than repaired it. Only the four pre-existing events gate the check now; the missing hook is drift, which MissingEntireHooks already reports with entire enable as the fix, and setupAgentHooks installs unconditionally so that fix still works.

The new hook was inert in E2E — found while fixing the above, not reported by either bot. The suite pre-trusts hooks by recomputing Codex's trusted_hash values itself (e2e/agents/codex_trust.go), from a hand-maintained event table that didn't know SessionEnd. Since Codex silently skips untrusted hooks, it would have been installed but never fired for every E2E run, with nothing failing to say so. The table learns SessionEnd, and TestCodexHookTrustState_CoversEveryInstalledEvent now installs hooks for real and requires a trust entry per Entire handler, so the two can't drift again.

Stale logger after the sweep (Bugbot + Copilot, same bug). logging.Close left the logger in place while closing the buffer underneath it, and the handler holds that *bufio.Writer by value — so lines logged after the sweep's cleanup went into an orphaned buffer over a closed file and were never flushed. entire doctor hits it directly: it keeps condensing and discarding sessions after the sweep returns. Close now drops the logger too. On its own that would push those lines back onto the terminal — the thing fe46a73 removed — so finalizeExitedSessions returns its cleanup instead of deferring it internally, and doctor scopes it to the whole command.

filterActiveSessions overclaimed parity with entire status (Copilot). Correct, and reachable rather than merely legacy: entire session attach sets Phase = PhaseEnded without stamping EndedAt, so an attached session showed as active in status while session stop — filtering on IsEnded — refused to list it. All three status sites now use IsEnded, which makes the comment true rather than merely reworded. (The divergence predates this PR; the diff only drew attention to it.)

Three regression tests plus the E2E guard: TestAreHooksInstalled_PreSessionEndInstall, TestEnsureInitialized_CleanupRestoresFallback, TestWriteActiveSessions_PhaseEndedWithoutEndedAtExcluded.

Testing

mise run check green: unit, integration, 60 canary E2E, 4 external-agent E2E. Plus the manual before/after against real codex exec described above. Re-run green on 1d36ecb67.

One flake observed on a first CI run — TestAttach_DiscoversExternalAgents and TestDiscoverAndRegister_Deduplication, both at exactly the 10s discovery timeout under load. Both pass in isolation, standalone, and on main; the re-run was fully green. Not caused by these changes, but noting it.

🤖 Generated with Claude Code

---

Note

Medium Risk
Touches session finalization, owner liveness, and interactive status/doctor sweeps; incorrect IDLE handling could finalize live sessions or leave checkpoint work uncondensed until PostCommit.

Overview
Fixes Codex (and similar) sessions staying active after the agent quits by wiring Codex SessionEnd into the normal finalize path and tightening the exited-owner safety net.

Codex: Installs and parses session-end (3s hook timeout, self-imposed ~2.5s SessionEndBudgeter on eager condense only). Hook install is refactored to a managedHooks table; hook sync now matches command + timeout so stale 30s SessionEnd entries get rewritten. Docs updated for Codex 0.146+ hooks and review caveats.

Lifecycle / reclaim: State.OwnerExited() now applies to IDLE as well as ACTIVE (only IsEnded() sessions are skipped). finalizeExitedSessions in entire status / entire doctor always marks candidates ENDED, but sweepCondenseBudget (1s total) caps eager condense so large backlogs do not stall status --json. endSessionNow takes an optional condense deadline; mark-ended is never budgeted. logging.EnsureInitialized routes sweep logs off the terminal; doctor classifies dead owners before phase heuristics.

Consistency: Active-session filtering uses IsEnded() in sessions / related paths.

^{Reviewed by Cursor Bugbot for commit 4cfc711. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 4cfc711. Configure here.}

[Copilot]

Pull request overview

Fixes un-finalized Codex sessions accumulating by wiring Codex’s SessionEnd hook into the lifecycle finalization path, and by extending the “owner process exited” safety net to cover IDLE sessions (not just ACTIVE), preventing lingering sessions and preserving pending checkpoint work.

Changes:

• Add Codex SessionEnd hook support (install, parse, budget) and finalize sessions on session end under a tight deadline.
• Broaden exited-owner reclamation to include IDLE sessions; introduce a sweep-wide condensation time budget to avoid stalling status/doctor.
• Add State.IsEnded() as the canonical “finalized session” predicate and align session-stop filtering with it.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
cmd/entire/cli/sessions.go	Uses IsEnded() for “active session” filtering and stop behavior.
cmd/entire/cli/session/state.go	Expands OwnerExited() to include IDLE; adds canonical IsEnded().
cmd/entire/cli/session/owner_test.go	Updates unit coverage for ended/EndedAt guard cases.
cmd/entire/cli/session/owner_live_test.go	Extends live-owner tests to cover IDLE + ACTIVE non-ended phases.
cmd/entire/cli/session_finalize.go	Adds sweep condense budget + ensures logging initialized during sweeps.
cmd/entire/cli/session_finalize_test.go	Covers IDLE exited-owner finalize + spent-budget still marks ENDED.
cmd/entire/cli/logging/logger.go	Adds EnsureInitialized() helper for non-hook commands.
cmd/entire/cli/logging/logger_test.go	Tests EnsureInitialized() init/no-op behavior.
cmd/entire/cli/lifecycle.go	Finalizes on SessionEnd with a condense-only deadline; adds deadline computation.
cmd/entire/cli/doctor.go	Improves stuck-session classification by prioritizing dead-owner detection.
cmd/entire/cli/doctor_classify_live_test.go	Adds platform-gated tests for IDLE + dead/alive owner classification.
cmd/entire/cli/agent/codex/types.go	Adds SessionEnd hook type support; generalizes session hook payload struct.
cmd/entire/cli/agent/codex/lifecycle.go	Registers SessionEnd hook + implements SessionEndBudgeter.
cmd/entire/cli/agent/codex/lifecycle_test.go	Tests shared SessionStart/SessionEnd parsing and budget constraints.
cmd/entire/cli/agent/codex/hooks.go	Refactors Codex hook management via a managed-hooks table; adds SessionEnd timeout matching.
cmd/entire/cli/agent/codex/hooks_test.go	Updates hook tests for SessionEnd + timeout rewrite behavior.
cmd/entire/cli/agent/codex/codex_test.go	Ensures session-end is advertised in Codex hook names.
cmd/entire/cli/agent/codex/AGENT.md	Updates Codex hook surface + documents SessionEnd constraints and review caveats.
cmd/entire/cli/agent/capabilities.go	Adds AsSessionEndBudgeter capability accessor.
cmd/entire/cli/agent/agent.go	Defines SessionEndBudgeter interface and documentation.
CLAUDE.md	Documents the updated exited-owner reclaim behavior and session-end budgeting.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[gtrrz-victor]

Code review

Reviewed the full branch (8 commits, 29 files, +1194/−298). Five findings, ordered by severity; the top three were confirmed against the code.

1. Condense deadline silently drops mirror checkpoint writes — medium

cmd/entire/cli/lifecycle.go:1141

The context.WithDeadline installed for the condense flows into CondenseAndMarkFullyCondensed → CondenseSession → store.Write. fanoutStore.Write (cmd/entire/cli/checkpoint/fanout.go:68) writes the primary first, then each mirror, treating mirror errors as best-effort (logging.Warn + drop).

Scenario: a Codex session with a large transcript quits; the primary v1 write finishes at ~2.4s, the 2.5s deadline expires, and every mirror write fails with context.DeadlineExceeded. Write still returns nil, so the session is marked FullyCondensed and nothing ever retries — the checkpoint is permanently missing from all configured mirrors, with only a log line to say so. The same shape applies to any context-aware step that runs after the primary ref update.

2. markSessionEnded is unbounded, so the write the budget exists to protect is the killable one — medium

cmd/entire/cli/lifecycle.go:1129

The doc comment says the mark-ended write "must always complete", but it runs before any deadline is installed, and acquireSessionGate (cmd/entire/cli/strategy/session_state.go:598) calls plain flock.Acquire — unbounded — unless sessionLockDeadlineFromContext is set, which only TurnStart opts into. MutateSessionState's own doc notes a callback "may hold the lock for slow operations… CondenseSession (shadow-branch tree builds, transcript compaction)".

Scenario: the user quits Codex immediately after a turn while the Stop hook's condense still holds the session gate; the acquire blocks past 3s, Codex SIGKILLs the process tree, and the session is never marked ENDED — exactly the orphaned-session failure this PR exists to eliminate. The 2.5s self-imposed budget never gets a chance to apply.

3. Doctor's deferred stopLogging is a no-op in the common case — low-medium

cmd/entire/cli/doctor.go:154

finalizeExitedSessions returns func(){} from its early exit (cmd/entire/cli/session_finalize.go:53) and only calls logging.EnsureInitialized after that guard.

Scenario: entire doctor with one time-based stuck session and zero exited sessions — the sweep returns (0, no-op), logging is never initialized, and the CondenseSessionByID / discardSession calls at doctor.go:187/195 emit structured slog lines to slog.Default() (the terminal), interleaved with doctor's own report. That is precisely the outcome the new comment at doctor.go:150-153 claims to prevent. Fix: hoist the logging setup above the early return, or move the guard into the caller.

4. Broadening OwnerExited to IDLE back-stamps EndedAt = now on sessions that ended days ago — low

cmd/entire/cli/session/state.go:528

markSessionEnded always sets state.EndedAt = &now. With IDLE sessions now in scope, the first entire status after upgrade finalizes every IDLE session whose agent has since exited (anything inside the 7-day StaleSessionThreshold). sessionLastActiveTime (resume_picker.go:196) prefers EndedAt, so a week-old walked-away session sorts to the top of the entire session resume picker ahead of yesterday's real work, and entire session info / session list (sessions.go:671) print "ended <now>" for a session whose agent died days earlier. Using the recorded last-interaction time when finalizing retroactively would avoid this.

5. E2E trust hash omits Codex's SessionEnd clamp — low

e2e/agents/codex_trust.go:100-107

The helper clamps only timeoutSec < 1, then hashes the raw configured value. This PR establishes that Codex clamps SessionEnd handlers to SESSION_END_MAX_TIMEOUT_SEC = 3. If that clamp is applied in Codex's normalized handler before command_hook_hash, any SessionEnd timeout above 3 in .codex/hooks.json produces a pre-trust hash that doesn't match, session_end stays untrusted, and the hook never fires for the whole e2e suite — the exact silent-inertness the new comment at line 51 warns about. TestCodexHookTrustState_CoversEveryInstalledEvent only checks event coverage, not the timeout. Benign today (Entire installs exactly 3), but worth mirroring the clamp so it stays benign.

[Soph]

Thanks — this was a genuinely useful pass. Verified all five against the code; three are fixed, one is latent, and one I resolved differently than suggested.

3. Doctor's deferred stopLogging is a no-op — fixed, and you're right about the fix

Confirmed exactly as described, and it was a gap in my own change from the previous commit. Took your first option: finalizeExitedSessions goes back to returning just int with its own internal teardown, and runSessionsFix now installs command-scoped logging itself through a new ensureCommandLogging helper (which pairs SetLogLevelGetter with EnsureInitialized — the pairing that was already being copy-pasted at every command-level Init site).

TestRunSessionsFix_HandlerLogsStayOffTheTerminal pins it. Without the fix it captures {"level":"INFO","msg":"session skipped: no transcript or files to condense",...} on slog.Default(), which is your scenario verbatim. Worth noting the first version of that test passed for the wrong reason — the discard path logs nothing — so it now drives the condense path and asserts the log file is non-empty.

4. EndedAt back-stamped to now — fixed

Confirmed. Checked your premise before changing anything: only display (sessions.go:671) and ordering (resume_picker.go:198) read the value and nothing keys retention off it, so an older timestamp is safe. Now an explicit endedAtPolicy — endedNow for the session-end hook and entire session stop, endedWhenLastSeen for the sweep — rather than keying off event == nil, so a future caller passing a nil event doesn't silently inherit back-stamping.

One thing the diff doesn't reveal: the obvious fix doesn't work. The EventSessionStop transition carries ActionUpdateLastInteraction (session/phase.go:179), so it overwrites LastInteractionTime with now before you can read it. My first attempt failed its own test for exactly that reason; the value is now captured before the transition runs.

5. Trust hash vs the SessionEnd clamp — resolved, but not by mirroring

I didn't mirror the clamp. Whether Codex applies it before command_hook_hash is unverified by both of us, and guessing wrong breaks trust for configs above the ceiling rather than preserving it — the failure mode being silent (an untrusted hook simply never fires) argues against guessing. Instead the helper documents precisely what is and isn't mirrored, and TestCodexHookTrustState_CoversEveryInstalledEvent now pins the premise that makes it moot: SessionEnd must be installed at exactly SessionEndTimeoutSec. Raise it and the test fails loudly instead of e2e going quietly blind.

1. Mirror writes dropped by the condense deadline — latent, not live

The mechanism is exactly as you describe, but it can't be reached in a shipped binary: the backend registry contains only the two git-backed backends, and Register — the path for a non-git mirror — is test-only by construction, so no production config can select a real mirror. Worth revisiting whenever a mirror backend actually ships, at which point it's really a question for fanoutStore (a mirror write that loses its deadline should be retried or surfaced regardless of who cancelled it) rather than for this PR.

2. markSessionEnded is the unbounded part — real, and sharpened by a second review

Confirmed, and stronger than you put it: WithSessionLockWait has exactly one caller in the tree (lifecycle.go:558, TurnStart), so every other path takes a blocking flock.

One correction on severity: a killed mark-ended doesn't orphan the session. The exited-owner sweep reclaims exactly that case on the next entire status / entire doctor — that's what the safety net is for. What was actually wrong was the doc comment claiming the write "must always complete", which I've rewritten to say that unbounded isn't the same as guaranteed and to name the sweep as the backstop.

An agent review on the trail landed on the same seam from the other direction and found the part that does cost something. Walking all three kill windows, the two you'd expect are safe, but one isn't: the condense commits the checkpoint to entire/checkpoints/v1 inside the MutateSessionState callback (manual_commit_condensation.go:434) while state is saved only after that callback returns (session_state.go:563). A kill in between leaves a committed checkpoint with CheckpointTranscriptStart / LastCheckpointID / StepCount / FullyCondensed all un-advanced, so PostCommit mints a fresh ID over the same transcript range. Duplication, not loss.

Taken the cheap half here: the self-imposed budget goes 2500ms → 2s. The headroom is not decoration — Codex's cap starts at the sh -c wrapper while ours starts at Go package init, so sh startup, the command -v PATH walk and loading a 66MB binary all land before our clock starts. Measured ~40ms warm, 120ms first run; a cold page cache or network filesystem is precisely where 500ms wasn't enough. The test now pins ≥1s of headroom so tightening it back has to be deliberate.

The durable fix — making the v1 write and the state save one recoverable unit — is a real change to the condensation path and doesn't belong in a session-cleanup PR, so it's filed as a follow-up.