fix: improve OpenRouter protocol interoperability

[nachiketb-nvidia]

What

• prevent inbound compression negotiation from being forwarded to upstream LLM providers
• accept optional [DONE] termination on Anthropic-compatible SSE streams
• preserve OpenAI Chat usage chunks that arrive after the finish chunk
• count terminal stream failures in JSON stats and OpenTelemetry errors

Why

OpenRouter exercises provider-compatible behavior that exposed four gaps: compressed responses the HTTP client could not decode, Anthropic streams ending with [DONE], usage arriving after finish_reason, and stream failures being reported as successful in stats.

How

The fixes stay at their existing ownership boundaries in the LLM client, translation codecs, and server usage observer. Routed calls remain counted once; terminal stream failures increment only the missing error counters.

What to review

• reserved-header handling for Accept-Encoding
• OpenAI Chat event ordering when usage follows the terminal choices chunk
• stream failure accounting without partial usage or latency

Validation

• cargo test -p switchyard-llm-client forwards_metadata_headers_except_reserved
• cargo test -p switchyard-translation
• cargo test -p switchyard-server streaming_error_records_error_without_usage_or_latency
• cargo clippy -p switchyard-llm-client -p switchyard-translation -p switchyard-server --all-targets -- -D warnings
• live OpenRouter: 9/9 buffered model-format combinations returned 200 with matching /v1/stats
• live OpenRouter streaming: Chat, Responses, and Anthropic all completed with matching usage and zero errors

Summary by CodeRabbit

• Bug Fixes
  • Streaming failures are now recorded accurately without inflating request, token, or latency metrics.
  • Caller-provided Accept-Encoding headers are no longer forwarded unexpectedly.
  • OpenAI streaming responses no longer include synthetic zero-valued usage data.
  • Usage received after a stop event is emitted in a dedicated usage-only update.
  • Completed streaming responses consistently include the [DONE] marker across supported formats.
• Tests
  • Expanded coverage for streaming errors, usage reporting, and completion markers.

[coderabbitai]

Walkthrough

The client reserves accept-encoding. The server records streaming failures in statistics and telemetry. OpenAI Chat translation separates usage chunks from terminal chunks. SSE termination emits [DONE] for supported formats.

Changes

Header forwarding

Layer / File(s)	Summary
Reserve accept-encoding crates/libsy-llm-client/src/client.rs	The client excludes caller-provided accept-encoding from upstream requests. Tests verify the header is absent.

Stream error metrics

Layer / File(s)	Summary
Record streaming failures crates/switchyard-server/src/stats/accumulator.rs, crates/switchyard-server/src/usage_metrics.rs, crates/switchyard-server/tests/server.rs	Streaming errors increment global and per-model error counts and the switchyard.errors metric. Request, token, and latency counts remain unchanged.

Stream translation

Layer / File(s)	Summary
Emit OpenAI usage chunks crates/switchyard-translation/src/codecs/openai_chat/stream.rs, crates/switchyard-translation/tests/stream_translation.rs	Completed streams emit backend usage in a separate empty-choices chunk. Terminal chunks omit usage when no backend usage was received.
Emit SSE terminal markers crates/switchyard-translation/src/sse.rs	done_marker returns [DONE] for supported wire formats, including Anthropic.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Poem

I’m a rabbit with metrics to count,
Headers stay home, not upstream-bound.
Usage hops in a final small chunk,
Errors are logged when streams have sunk.
[DONE] now marks the closing sound.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the pull request's main goal of improving OpenRouter protocol interoperability.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🧹 Nitpick comments (1)

crates/switchyard-server/tests/server.rs (1)

1133-1133: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Document the important stream-failure contract.

Add a short comment stating that a terminal stream failure increments error statistics and metrics without recording usage or terminal latency.

As per coding guidelines, “comments for ... important tests” are required.

Proposed comment

+// A terminal stream failure records errors without usage or terminal latency.
 async fn streaming_error_records_error_without_usage_or_latency() -> TestResult {

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/switchyard-server/tests/server.rs` at line 1133, Above the
streaming_error_records_error_without_usage_or_latency test, add a brief comment
documenting that terminal stream failures increment error statistics and metrics
while recording neither usage nor terminal latency.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/switchyard-server/src/usage_metrics.rs`:
- Around line 94-102: Normalize the optional tier value at the start of
record_stream_error, trimming whitespace and converting empty values to None,
then pass the normalized tier to both StatsAccumulator::record_stream_error and
attributes. Apply the same normalization consistently in the related token and
latency metric recording paths that currently pass the original tier.

In `@crates/switchyard-translation/src/sse.rs`:
- Around line 17-22: Update done_marker and the decode_stream flow so data:
[DONE] produces a distinct terminal result instead of being treated as an empty
frame. Detect the marker in parse_json_sse_frame or its caller, return the
marker-specific result, and break decoding immediately while preserving
empty-frame skipping and normal frame processing.

---

Nitpick comments:
In `@crates/switchyard-server/tests/server.rs`:
- Line 1133: Above the streaming_error_records_error_without_usage_or_latency
test, add a brief comment documenting that terminal stream failures increment
error statistics and metrics while recording neither usage nor terminal latency.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 036c77bf-f046-4ebe-b59d-f6648bdfd457

📥 Commits

Reviewing files that changed from the base of the PR and between d363dfc and c70579a.

📒 Files selected for processing (7)

• crates/libsy-llm-client/src/client.rs
• crates/switchyard-server/src/stats/accumulator.rs
• crates/switchyard-server/src/usage_metrics.rs
• crates/switchyard-server/tests/server.rs
• crates/switchyard-translation/src/codecs/openai_chat/stream.rs
• crates/switchyard-translation/src/sse.rs
• crates/switchyard-translation/tests/stream_translation.rs