feat: expose provider request ids on STT/TTS/LLM spans for debugging#5546
Conversation
Adds a new span attribute 'lk.provider_context_ids' (list[str]) populated from plugin-provided STT request_ids and TTS segment_ids, so users can cross-reference traces with provider-side logs when debugging. - TTS: AudioEmitter.start_segment appends non-empty segment_ids to a deduped list and sets the attribute on the current tts_request_run span. - STT: AudioRecognition collects SpeechEvent.request_ids across a turn and writes the full list on the user_turn span at span end.
- tts base: add AudioEmitter.note_provider_context_id() for plugins where the provider id arrives after start_segment() has been called. - sarvam TTS: feed the captured server request_id via note_provider_context_id once the first WS response arrives. - sarvam STT: fall back to the session-wide server request_id when a per-message request_id is missing. - openai STT (realtime): populate SpeechEvent.request_id from item_id; track current_item_id for interim transcripts. - upliftai TTS: pass segment_id as the request id to client.synthesize() so the id is actually known to the server.
LLMStream now tracks the deduped list of ChatChunk.id values seen during each attempt and writes them to the llm_request_run span as lk.provider_context_ids, matching the per-attempt scope used for TTS. The list is reset in _main_task at the start of each attempt, populated by the monitor task as chunks arrive, and read at the end of the attempt (before the span closes). No plugin changes needed — ChatChunk.id is already the provider's response id across all LLM plugins.
Now that the attribute covers STT (request_id), LLM (ChatChunk.id, also a request/response id), and TTS (context_id / session_id), 'context_ids' was only accurate for TTS. 'request_ids' matches the terminology STT and LLM providers actually use and reads naturally for TTS context_ids too. Renames: - ATTR_PROVIDER_CONTEXT_IDS -> ATTR_PROVIDER_REQUEST_IDS - lk.provider_context_ids -> lk.provider_request_ids - AudioEmitter.note_provider_context_id() -> AudioEmitter.note_provider_request_id() - _provider_context_ids / _stt_context_ids -> _provider_request_ids / _stt_request_ids
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| finally: | ||
| if self._provider_request_ids: | ||
| attempt_span.set_attribute( | ||
| trace_types.ATTR_PROVIDER_REQUEST_IDS, self._provider_request_ids | ||
| ) |
There was a problem hiding this comment.
🟡 LLM _provider_request_ids may be incomplete when set on attempt span due to race with monitor task
In llm.py, _provider_request_ids is populated by the _metrics_monitor_task (line 294-295) which runs as a separate asyncio task, and read in the finally block of _main_task (line 228-231). When _run() returns, the finally block executes synchronously before the event loop can schedule the monitor task. Any ChatChunk events that were send_nowait'd by _run() after its last internal await but before returning will be buffered in the tee iterator and not yet processed by the monitor task. This means the attempt_span's lk.provider_request_ids attribute could be incomplete. In practice, since LLM request IDs are typically identical across all chunks, the first processed chunk captures the ID, making this unlikely to cause missing data — but it's architecturally fragile.
Was this helpful? React with 👍 or 👎 to provide feedback.
|
This is an automated Claude Code Routine created by @toubatbrian. Right now it is in experimentation stage. The automation will start porting this PR into agents-js automatically. Tracking: porting Generated by Claude Code |
4 participants
Adds a new span attribute
lk.provider_request_ids(list[str], deduped) on theuser_turn(STT),tts_request_run(TTS), andllm_request_run(LLM) spans for debugging/reporting STT or TTS issues to the provider.Supported plugins (id actually known to the provider)
For LLM, the id comes from
ChatChunk.id.TTS streaming (
tts_request_runattribute populated)note_provider_request_id()STT (
user_turnattribute populated when present)metadata["request_id"]session.request_iditem_idfrom completion event