fix(llm-client): enable Anthropic prompt caching by default

[nachiketb-nvidia]

What

Enable Anthropic prompt caching by default in TranslatingLlmClient.

Why

Native Anthropic requests require an explicit cache breakpoint to use provider prompt caching.

How

For Backend::Anthropic, the client marks the final message content block with {"cache_control": {"type": "ephemeral"}} after request translation. Other backend formats are unchanged.

What to review

• caching is limited to Anthropic backends
• existing explicit cache control on the final block is preserved
• protocol and translation types remain unchanged

Validation

• cargo test -p switchyard-llm-client anthropic_prompt_caching_marks_final_message
• cargo clippy -p switchyard-llm-client --all-targets -- -D warnings

[coderabbitai]

Walkthrough

The change adds request-level caching control to LlmRequest and applies Anthropic prompt-caching rules to encoded request bodies. Tests cover default values, explicit disabling, marker injection, and marker removal.

Changes

Prompt caching

Layer / File(s)	Summary
Request caching contract crates/protocol/src/llm.rs	LlmRequest adds a public caching field. Manual defaults enable caching. Deserialization preserves explicit values.
Anthropic cache policy crates/libsy-llm-client/src/client.rs	Anthropic processing removes cache markers when caching is disabled and preserves or injects ephemeral markers when caching is enabled. Tests cover injection and removal.

Estimated code review effort: 4 (Complex) | ~45 minutes

Poem

A rabbit hops through cached text,
And marks the final block with care.
If caching sleeps, the marks depart,
While defaults keep the feature there.
“Thump-thump!” says Bun, “the paths are clear!”

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: enabling Anthropic prompt caching by default.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🧹 Nitpick comments (1)

crates/libsy-llm-client/src/client.rs (1)

936-952: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Broaden test coverage for the new caching branches.

The new test only covers a single plain-string message content block being marked and then unmarked. It does not exercise: preserving a caller-supplied cache_control when caching is already enabled (the has_anthropic_cache_control early return), the system-prompt fallback, the last-tool fallback, or removal from a tools array with input_schema. Given this logic gates a cost-sensitive feature enabled by default, add cases for these branches.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/libsy-llm-client/src/client.rs` around lines 936 - 952, Expand the
test `anthropic_prompt_caching_is_injected_and_can_be_disabled` with cases
covering preservation of caller-supplied `cache_control` via the
`has_anthropic_cache_control` early return, system-prompt fallback, last-tool
fallback, and removal of `cache_control` from tools containing `input_schema`.
Assert each branch’s expected JSON behavior while retaining the existing
plain-string injection and disablement coverage.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/libsy-llm-client/src/client.rs`:
- Around line 718-779: Add concise comments to the private helpers
mark_last_text_cacheable, has_anthropic_cache_control, remove_cache_controls,
and is_anthropic_cacheable explaining their traversal purpose, the meaning of
Anthropic-cacheable content, the reverse traversal used to select the last text
block, and why input_schema is excluded from recursion. Keep the implementation
unchanged and align the wording with the nearby apply_anthropic_prompt_caching
comment.
- Around line 685-740: Update apply_anthropic_prompt_caching and
mark_last_text_cacheable to identify and mark the last block accepted by
is_anthropic_cacheable, including trailing image, document, tool_use, and
tool_result blocks rather than only text blocks. Add tests covering trailing
tool_use and tool_result blocks, and ensure the implementation does not create
redundant manual cache breakpoints when supported automatic top-level
cache_control is used.

---

Nitpick comments:
In `@crates/libsy-llm-client/src/client.rs`:
- Around line 936-952: Expand the test
`anthropic_prompt_caching_is_injected_and_can_be_disabled` with cases covering
preservation of caller-supplied `cache_control` via the
`has_anthropic_cache_control` early return, system-prompt fallback, last-tool
fallback, and removal of `cache_control` from tools containing `input_schema`.
Assert each branch’s expected JSON behavior while retaining the existing
plain-string injection and disablement coverage.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 58c8b986-f59b-421b-918f-54e783240d64

📥 Commits

Reviewing files that changed from the base of the PR and between d676965 and 21f83ad.

📒 Files selected for processing (2)

• crates/libsy-llm-client/src/client.rs
• crates/protocol/src/llm.rs

[ayushag-nv]

lgtm