Skip to content

[bot] Cohere: Classify API (client.classify()) not instrumented #343

@braintrust-bot

Description

@braintrust-bot

Summary

The Cohere Classify API (client.classify()) is not instrumented. Calls to client.classify() (sync and async) produce zero Braintrust tracing. This is an active, GA inference endpoint that accepts text inputs with example text+label pairs and returns classification predictions with confidence scores.

The Cohere integration currently instruments chat, chat_stream, embed, and rerank (V1 and V2), but has no handling for the classify() surface. This gap was explicitly noted as a follow-up when the Cohere integration was implemented (see #260).

This repo already instruments analogous classification/moderation endpoints for other providers:

What is missing

Cohere Resource Method Instrumented?
client.chat() / client.v2.chat() sync + async Yes
client.chat_stream() / client.v2.chat_stream() sync + async Yes
client.embed() / client.v2.embed() sync + async Yes
client.rerank() / client.v2.rerank() sync + async Yes
client.classify() sync No
client.classify() async (AsyncBaseCohere) No

API details

  • Endpoint: POST /v1/classify
  • Purpose: Few-shot text classification — predicts which label best fits specified text inputs using provided text+label example pairs as reference
  • Supports: Up to 96 texts per request, custom fine-tuned models (no examples required), confidence scores, multi-label classification
  • Parameters: inputs, examples, model, preset, truncate
  • Return type: ClassifyResponse with classifications array containing id, input, predictions, prediction, confidence, confidences, labels

Minimum instrumentation

At minimum, both sync and async classify() methods should create spans capturing:

  • Input: inputs (texts being classified), examples (reference text+label pairs)
  • Output: classifications array with predictions and confidence scores
  • Metadata: provider: "cohere", model, preset, truncate setting, number of unique labels
  • Metrics: latency

Precedent in this repo

  • OpenAI ModerationsPatcher instruments the equivalent client.moderations.create() endpoint
  • Mistral classifiers gap ([bot] Mistral: Classifiers API (client.classifiers.moderate() / classify()) not instrumented #326) tracks the same pattern: accept input text, run a classifier/moderation model, return inference results
  • The existing Cohere patchers (ChatPatcher, EmbedPatcher, RerankPatcher) in py/src/braintrust/integrations/cohere/patchers.py provide the exact pattern to follow: FunctionWrapperPatcher targeting BaseCohere.classify and AsyncBaseCohere.classify

Braintrust docs status

not_found — Cohere is not listed on the Braintrust integrations page. No dedicated Cohere integration docs page mentions classify.

Upstream sources

  • Cohere Classify API reference: https://docs.cohere.com/reference/classify
  • Cohere Python SDK on PyPI (v6.1.0): client.classify() on BaseCohere and AsyncBaseCohere
  • The classify endpoint is listed in the main Cohere API reference alongside chat, embed, and rerank — it is not deprecated

Local files inspected

  • py/src/braintrust/integrations/cohere/patchers.py — defines patchers for Chat, ChatStream, Embed, Rerank (V1 and V2, sync and async); zero references to classify
  • py/src/braintrust/integrations/cohere/tracing.py — wrapper functions for chat, embed, rerank; no classify wrappers
  • py/src/braintrust/integrations/cohere/integration.py — integration class registers 4 composite patchers; no ClassifyPatcher
  • py/src/braintrust/integrations/cohere/test_cohere.py — no classify test cases
  • py/pyproject.toml — Cohere version matrix: latest pinned to cohere==6.1.0, older pin at 5.0.0

Relationship to existing issues

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions