Dogfood benchmark harness for speaker ID

[ComputelessComputer]

Draft harness for measuring speaker identification quality. Part of #1.

What changed (pivoted from synthetic to dogfood)

Dropped the VoxCeleb/LibriSpeech stitcher. Replaced with a labeler that runs on your own Char meetings.

Rationale: synthetic meetings optimize the wrong thing. Read speech doesn't sound like a meeting. Celebrity interviews aren't in Korean. AMI is closer but still not Char conditions. Your last 20 meetings are the actual target distribution — label them in an evening, measure against that.

Pipeline

your meeting-*.md exports
     ↓  label.py (play 5s per speaker, type a name)
ground/<meeting_id>.json
     ↓  extract_embeddings.py (TODO — Swift bridge)
meetings/<meeting_id>/embeddings.json
     ↓  score.py (Python mirror of store.ts)
results/<name>.json
     ↓  report.py
accuracy · unknown_rejection · false_accept · calibration_error

Why four metrics, not one

store.ts has a 0.04 margin gate specifically to reject unknown speakers. A benchmark that only tracks identification accuracy pushes the optimizer to drop that gate and confidently mislabel every new person. score.py splits each meeting's humans in half — enrolled vs strangers — so regressions on rejection get caught.

What's still missing

• extract_embeddings.py — the Swift speaker-embedding extractor needs a CLI entry point. Until that lands, embeddings.json doesn't get generated. Tracked in Build the speaker ID benchmark dataset (dogfood 20 meetings) #3.
• baseline.json — populated once the pipeline runs end-to-end.
• Raw meeting export — if speakers are already labeled in the app, the markdown has human names rather than Speaker 1/Speaker 2, and matcher guesses contaminate ground truth. Either label fresh meetings or add uchar meetings export --raw.

Status

Draft. Safe to land the labeler and scorer now; embedding extraction and baseline happen next.

Part of #1 and #3.