feat: Move observation from libsy to libsy-llm-client

[grahamking]

The client is in a better position to time the call, because it makes
the call. That gives us more accurate stats and locates the observation
in the natural place. Finally it means you can use run_stream without
having to implement an observer.

Mostly a straight code move.

Now supports timing multiple concurrent overlapping LLM calls. Thanks Opus
for catching that!

I also renamed testing function drive to test_drive, because it
clashed with regular function drive. And a minor metrics:: import
change for readability.

Part of #310

Assisted-by: Codex:GPT 5.6 Sol medium
Reviewed-by: Claude:Opus 5 medium

Signed-off-by: Graham King grahamk@nvidia.com

Summary by CodeRabbit

• New Features
  • Added run-level observability for LLM calls, including selected model, routing status, success, duration, usage, and routing overhead.
  • Exposed observation types for integrations that monitor completed runs.
• Metrics
  • Improved routing-overhead measurement, including overlapping and failed calls.
• Documentation
  • Clarified routing-overhead metric definitions and troubleshooting guidance.
• Refactor
  • Simplified algorithm execution and testing interfaces without changing routing behavior.

[grahamking]

@CodeRabbit review

[github-actions]

PR Preview Action v1.8.1
🚀 View preview at https://NVIDIA-NeMo.github.io/Switchyard/pr-preview/pr-332/
Built to branch gh-pages at 2026-08-07 19:38 UTC. Preview will be ready when the GitHub Pages deployment is complete.

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Walkthrough

The change moves request-scoped observations into libsy-llm-client. It measures routed provider-call time, emits LLM-call and routing-overhead events, simplifies the core algorithm API, updates test helpers, and aligns server imports and metric documentation.

Changes

Run observability migration

Layer / File(s)	Summary
Observation contracts and metrics crates/libsy-llm-client/Cargo.toml, crates/libsy-llm-client/src/{lib,metrics,observation,client}.rs	The client exports observation types, records routing overhead, and uses module-qualified metric helpers.
Client run observation flow crates/libsy-llm-client/src/run.rs, crates/libsy-llm-client/tests/observability.rs	Runs measure provider calls, merge overlapping successful routed-call intervals, emit observations, and test the resulting events.
Core driver API simplification crates/libsy/src/core/{algorithm,testing}.rs, crates/libsy/src/{lib,observability}.rs	The core driver removes observer state and routing timing. Stream and drive APIs no longer accept observers. Tests use test_drive.
Algorithm test harness migration crates/libsy/src/algorithms/*.rs	Algorithm tests replace drive with test_drive without changing routing scenarios or assertions.
Server and metric definition alignment crates/switchyard-server/src/lib.rs, docs/internal/metrics_reference.md	The server imports observation types from the LLM client. Metric documentation defines total-run timing, successful routed calls, failed attempts, and overlapping calls.

Estimated code review effort: 4 (Complex) | ~45 minutes

Poem

I’m a rabbit watching timers run,
Calls hop neatly, one by one.
Overlaps merge beneath the moon,
Observations arrive in tune.
The driver sheds its watchful ears—
Clean new paths for coming years!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: moving observation functionality from libsy to libsy-llm-client.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/libsy-llm-client/src/run.rs`:
- Around line 181-195: Capture the end timestamp immediately after
RoutedLlmClient::call completes, before invoking
observability::observe_client_call(result), and compute the duration from that
timestamp. Keep the subsequent observation and result handling unchanged so
LlmCallObservation::duration measures only the client call.

In `@docs/internal/metrics_reference.md`:
- Line 60: Update the `switchyard_routing_overhead_ms` documentation to state
that successful algorithm completion is required for recording, or move the
`RoutingOverhead` emission in `run` before propagating errors from
`drive(...).await?`. Ensure runs with a successful routed call followed by an
algorithm or concurrent-serving error still record the overhead metric.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 89d1d284-9904-41a1-bbc0-a9a95483133c

📥 Commits

Reviewing files that changed from the base of the PR and between 981ca62 and 405cb38.

📒 Files selected for processing (21)

• crates/libsy-llm-client/Cargo.toml
• crates/libsy-llm-client/src/client.rs
• crates/libsy-llm-client/src/lib.rs
• crates/libsy-llm-client/src/metrics.rs
• crates/libsy-llm-client/src/observability.rs
• crates/libsy-llm-client/src/observation.rs
• crates/libsy-llm-client/src/run.rs
• crates/libsy-llm-client/tests/observability.rs
• crates/libsy/src/algorithms/fall_through.rs
• crates/libsy/src/algorithms/llm_class.rs
• crates/libsy/src/algorithms/noop.rs
• crates/libsy/src/algorithms/passthrough.rs
• crates/libsy/src/algorithms/rand.rs
• crates/libsy/src/algorithms/stage.rs
• crates/libsy/src/algorithms/subagent_affinity_tests.rs
• crates/libsy/src/core/algorithm.rs
• crates/libsy/src/core/testing.rs
• crates/libsy/src/lib.rs
• crates/libsy/src/observability.rs
• crates/switchyard-server/src/lib.rs
• docs/internal/metrics_reference.md

💤 Files with no reviewable changes (1)

• crates/libsy-llm-client/src/observability.rs

[ayushag-nv]

lgtm ! RoutedCallWindows approach is cool