fix(soniox): emit PREFLIGHT_TRANSCRIPT for preemptive LLM generation#5553
Merged
davidzhao merged 1 commit intoApr 25, 2026
Merged
Conversation
When a Soniox token batch contains only final tokens (no non-final/speculative tokens), speech has reached a stable state indicating a likely pause. Emit PREFLIGHT_TRANSCRIPT instead of INTERIM_TRANSCRIPT in this case to allow AgentSession to start preemptive LLM generation, matching the latency benefit already available via Deepgram v2's EagerEndOfTurn event. Fixes livekit#5536
|
octo-patch seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
davidzhao
approved these changes
Apr 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5536
Problem
The Soniox STT plugin only emits
INTERIM_TRANSCRIPTevents, soAgentSession's preemptive LLM generation never fires when using Soniox for STT-driven end-of-turn detection. Users pay full LLM TTFT in wall-clock time after every turn, which is too high for real-time voice use cases.Solution
When a Soniox token batch contains only final tokens (no non-final/speculative tokens pending), speech has reached a stable state — this indicates a likely pause, analogous to Deepgram v2's
EagerEndOfTurnsignal. In that case, emitPREFLIGHT_TRANSCRIPTinstead ofINTERIM_TRANSCRIPT.This allows
AgentSessionto start speculative LLM generation. If the user resumes speaking, the next batch will contain non-final tokens, triggering a regularINTERIM_TRANSCRIPTthat aborts the preemptive generation — the same abort path used by Deepgram v2'sTurnResumedevent.The change is minimal: one conditional in the event-type selection within
_recv_messages_task.Testing
Tested by code review against the Deepgram v2 pattern (
EagerEndOfTurn→PREFLIGHT_TRANSCRIPT). The logic mirrors that behavior using the token finality heuristic native to Soniox's streaming API.