[Enhancement]: Support Oracle Cloud Infrastructure (OCI) Generative AI as a provider #548

Affected area

Rust core runtime, Python binding, Node.js binding, Framework integrations, Observability or exporters, Documentation or examples

Problem or opportunity

NeMo Relay ships provider codecs for OpenAI Chat, OpenAI Responses, and Anthropic Messages, so middleware (PII redaction, guardrails, policy) and observability subscribers can reason over those providers' payloads through the normalized annotated shapes. There is currently no coverage for Oracle Cloud Infrastructure (OCI) Generative AI, whose chat API uses its own request envelope (ChatDetails with servingMode and chatRequest) and its own wire formats (GENERIC, COHERE, and COHEREV2).

OCI Generative AI serves managed on-demand models (Meta Llama, Cohere Command, xAI Grok, Google Gemini, OpenAI) and dedicated AI cluster endpoints, including imported open-weights models such as NVIDIA Nemotron 3 — so agents calling those models today are invisible to Relay's request intercepts and get no normalized response annotations.

Proposed enhancement

Add OCI Generative AI as a supported provider surface, delivered as a four-part stacked series (plan of record in the comments below, each layer independently green):

1. feat(codec): add OCI Generative AI provider support #554 — typed ApiSpecific request/response variants and an LlmResponseCodec decoding GENERIC, COHERE, and COHEREV2 REST responses (content, tool calls, finish reasons, usage, unmodeled-field preservation).
2. Request-side LlmCodec with encode(decode(x), x) == x identity and merge-not-replace encoding, so intercepts (redaction, policy) can safely edit OCI requests.
3. ProviderSurface::OCIGenAI registration and detection, an OCIGenAIStreamingCodec (SSE for all formats), and provider awareness in guardrails, the PII-redaction overlay, and adaptive components.
4. Python and Node.js binding exposure (pyo3 + napi classes, .pyi/codecs.py), with binding tests.

A supported-integrations guide for observing LangChain agents backed by langchain-oci follows separately if it has genuine value-add beyond examples. Follow-ups under discussion with the maintainers: OCI model pricing entries for cost accounting, and Switchyard routing support.

Runtime contract and binding impact

The Rust core is the source of truth: codecs live in crates/core/src/codec with typed variants in crates/types, and accept the documented camelCase REST wire format only (alternate renderings from OCI tooling are the caller's responsibility). Python and Node.js get codec classes mirroring the existing built-in codec exposure (layer 4); Go and the raw C FFI are unaffected (experimental, source-first). Managed-execution semantics are unchanged — the codecs plug into the existing middleware and event pipeline, and responses annotate LLMEnd events exactly like the other providers.

Alternatives considered

• Python-level integration only (Add OCI Generative AI provider codecs and LangChain integration guide #549): closed per review — the runtime contract requires provider behavior in the Rust core with binding parity.
• One monolithic PR (feat(codec): add OCI Generative AI provider codec in Rust core #552): closed in favor of this reviewable stacked split; the series recombines byte-identically to the validated original.
• Routing GENERIC through the existing OpenAI codec: rejected — the wire is OpenAI-inspired but not compatible (flat tool calls without the nested function object, ChatResult envelope, uppercase enums, different usage keys); the OpenAI codec hard-errors on it.
• Native Cohere V1/V2 codecs instead of an OCI codec: rejected — Oracle's CohereChatResponse wire schema (camelCase, chatResponse envelope, no generation_id/response_id) is not Cohere's own API; a native Cohere codec would decode none of it.

Acceptance criteria

• Relay decodes live OCI GENERIC, COHERE, and COHEREV2 chat responses into normalized annotations (message, tool calls, finish reason, usage), with unmodeled fields preserved in extra.
• Request decode/encode satisfies encode(decode(original), original) == original with unmodeled-field preservation, verified byte-identically against live captures.
• OCI is auto-detected as a provider surface and streaming responses are handled for all formats.
• Python and Node.js expose the codec with tests; docs cover the provider in the supported list.
• Every layer passes the full workspace gate (build, tests, clippy, fmt) independently, and behavior is validated against the live service (all model families, tool calls, finish-reason and usage variants).