Skip to content

🤖 feat: add opt-in AgentPond tracing - #3772

Open
marcusschiesser wants to merge 2 commits into
coder:mainfrom
marcusschiesser:agent/agentpond-tracing-20260731
Open

🤖 feat: add opt-in AgentPond tracing#3772
marcusschiesser wants to merge 2 commits into
coder:mainfrom
marcusschiesser:agent/agentpond-tracing-20260731

Conversation

@marcusschiesser

Copy link
Copy Markdown

Summary

  • add opt-in AgentPond/OpenInference tracing to Mux's real Node StreamManager → streamText path
  • keep tracing disabled unless a Files SDK provider is explicitly configured
  • redact prompt and response content while retaining operation, model, timing, and token-usage metadata
  • document the privacy/configuration behavior and update generated built-in skill content

Validation

  • bun install --frozen-lockfile
  • make static-check (all project checks passed; Hadolint was rerun with the CI-pinned v2.12.0 because Homebrew v2.15 adds a warning for an unchanged Dockerfile line)
  • bun test src/node/services/streamManager.test.ts src/node/services/streamManager.modelOnlyNotifications.test.ts — 105 passed, 1 existing credential-gated test skipped
  • end to end: invoked the actual private StreamManager.createStreamResult path with AI SDK's MockLanguageModelV3, consumed the stream, synced the local Files SDK store, and read trace 970445ba8cfaea3856d3d6c1bcbbaac5 back
    • generation b6b86e88b4319dd1, provider mux-agentpond-e2e, model fixture-model
    • 7 input / 4 output / 11 total tokens
    • trace and observation content were null; raw-store search found neither prompt nor response sentinel

Configuration and limitations

AgentPond tracing is independent of Mux's anonymous usage telemetry. It activates only when FILES_SDK_PROVIDER and the selected Files SDK provider variables are present before startup. The integration intentionally records no prompt or response bodies.


Generated with mux • Model: codex/gpt-5 • Thinking: high • Cost: $unknown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 93bafb8214

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/node/services/agentPondTracing.ts Outdated
Comment on lines +52 to +54
onEnd: flushAgentPondTracing,
onAbort: flushAgentPondTracing,
onError: flushAgentPondTracing,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Restrict forced flushes to short-lived CLI shutdown

When tracing uses a remote or slow Files SDK provider, these async integration hooks force an export after every successful, aborted, or failed turn, so stream completion can wait for network I/O or the exporter's timeout. Because the same telemetry object is attached by the shared StreamManager, long-lived Electron and server sessions also pay this CLI-durability cost and receive no batching benefit; force-flush only when an actual short-lived CLI process is exiting, or make the per-turn flush bounded and non-blocking.

AGENTS.md reference: AGENTS.md:L10-L10

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in a3e57c7. The AI SDK integration no longer registers per-turn onEnd / onAbort / onError flush hooks, so normal long-lived turns use the batch processor as intended; the explicit lifecycle/test flush helper remains available. The 105 focused tracing and stream-manager tests pass (1 existing credential-gated skip). A fresh production StreamManager.createStreamResult E2E run completed with the AI SDK mock model, then flushed at the validation lifecycle boundary and read back 1 object / 4 events: trace 7da03f7d77d9cc673d3d153ff9bc9cf3, generation 2ebfce5734f2b1c1, with null trace/observation input/output and no prompt/response sentinels in the raw store. make static-check also passed ESLint, both TypeScript targets, formatting, generated-content, eager-import, code-link, and shell checks before the local Homebrew Hadolint 2.15.0 rejected an unchanged Dockerfile line that the repo's pinned 2.12.0 accepts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant