recipe(deberta): add mixedbread-ai/mxbai-rerank-base-v1 (text-classification, reranker)

[DingmaomaoBJTU]

Summary

Adds a curated CPU float recipe for mixedbread-ai/mxbai-rerank-base-v1 — a cross-encoder reranker that scores (query, document) relevance (config model_type = deberta-v2, DebertaV2ForSequenceClassification, num_labels = 1 → a single relevance logit). Ships fp32 + fp16 variants under cpu/cpu/ (both quant: null; no CPU quantized variant per repo convention).

This is a recipe-only (L0★) contribution: Optimum already covers deberta-v2 text-classification natively, so main builds this model with zero source changes. The delta is the curated CPU reference recipe plus an L2 numeric + ranking parity proof that the trained reranker head is preserved end-to-end. Notably the recipe is byte-identical to the now-merged cross-encoder/nli-deberta-v3-base recipe (#1117, on main at e95011d3) — identical git blob OID 7b36e69520b418c7c86b2d8a39eab741cb9742c5 (git-stored SHA256 8b587c4b…bec8; verify with git rev-parse HEAD:<path>). The model_id is CLI-passed and the output width is config-driven, so one deberta-v2 text-classification recipe serves the whole task family. Claimed tiers: Effort L0★ · Goal ceiling L2 · Outcome L0.

Updated after review (#1118): rebased onto current origin/main (e95011d3, which now includes the merged #1117 NLI recipe) and corrected the byte-identity evidence to git's canonical blob OID. (An earlier Get-FileHash value 59E337… was the Windows CRLF working-tree hash; git normalizes to LF on commit, so the on-main content hash is 8b587c4b….)

---

1. Recipe path(s)

• examples/recipes/mixedbread-ai_mxbai-rerank-base-v1/cpu/cpu/text-classification_fp32_config.json
• examples/recipes/mixedbread-ai_mxbai-rerank-base-v1/cpu/cpu/text-classification_fp16_config.json

Byte-identical (quant: null float bucket; on CPU both realize as fp32, fp16 materializes on GPU/NPU). opset 17, batch 1, inputs input_ids[1,512] + attention_mask[1,512] (int32), output logits[1,1]. No token_type_ids — deberta-v2 has type_vocab_size = 0.

2. README row

None. Recipe-only, CPU-only — deliberately not added to the "passes fp16 eval on all 10 (EP, device) buckets" table, which would be a factual overclaim for a CPU-only recipe (consistent with #1084 / #1112 / #1117).

3. Build output dir

temp/mxbai_rerank/ (scratch, gitignored) — model.onnx + model.onnx.data (786 MB fp32).

4. Build log

✅ Build complete in 633.6s (Export 369.6s + Optimize 256.9s; no quantize — quant: null). ONNX IR 8, opset 17, external data co-located, output logits[1,1].

5. Appended findings

model_knowledge/deberta.json → deberta-002 (second model in the deberta family; mxbai-rerank-base-v1 added to models_tested). Lane A (skill repo); not part of this model PR's diff.

6. Optimum-coverage probe

deberta-v2 text-classification is VENDOR-ONLY (added_by_winml = []). Vendor onnx tasks: [feature-extraction, fill-mask, multiple-choice, question-answering, text-classification, token-classification]. No winml-added exporter exists or is needed.

7. Claimed (Effort, Goal, Outcome)

L0★ / L2 / L0. Baseline: winml build -m mixedbread-ai/mxbai-rerank-base-v1 on main (origin/main = e95011d3 — branch rebased onto current main; winml 0.2.0) already PASSES (default pipeline 973.3s incl. quantize → uint8/16 327.5 MB). Contribution = curated CPU float recipe (fp32 + fp16) + L2 relevance-head-preservation proof.

8. Goal-ladder verdict table

Tier	Verdict	Evidence
L0 (build)	PASS	recipe-driven float build exit 0 (633.6 s, no quantize); ONNX opset 17, inputs input_ids[1,512]+attention_mask[1,512], output logits[1,1], external data co-located
L1 (perf)	PASS	CPU/fp32 avg 7027.85 ms, 0.14 samples/s, RAM Δ +624.7 MB (also auto-ran on QNN/NPU at 523.99 ms / 1.91 samples/s)
L2 (numeric vs PyTorch)	PASS	4 real (query, doc) pairs: cosine 1.000000, max-abs 1.097e-05, ranking order 4/4 identical, top-1 doc agrees

Ceiling L2 reached; no downgrade.

9. Methodology-evolution declaration

No methodology friction. This is the second model in the deberta family (added in #1117) — it exercised the self-learning path as designed: deberta-001 predicted the recipe shape, the recipe came out byte-identical (same git blob OID 7b36e695…), and deberta-002 records the reranker-specific ranking-parity check. No skill_meta change needed.

10. Perf & eval data

EP / Device	Precision	Verdict	Mean	p50	Throughput	RAM Δ	Task metric
CPUExecutionProvider / cpu	fp32	PASS	7027.85 ms	6933.79 ms	0.14 samples/s	+624.7 MB	N/A (L3 not marched — L2 ceiling)
QNNExecutionProvider / npu	(auto)	PASS (bonus)	523.99 ms	522.76 ms	1.91 samples/s	+1791.7 MB	— (not a full NPU validation)

winml eval not run (Goal ceiling L2). The NPU row is a bonus data point from winml perf auto-selecting QNN — it is not a claimed coverage bucket (no NPU-side L2 parity yet), so coverage stays CPU-only/partial.

11. Component / op-level data

winml analyze --ep all: 568 total operators, 17 unique (identical op profile to the NLI sibling). Per-EP: QNN NPU 17/17 supported, OpenVINO NPU 17/17 supported (includes the disentangled-attention GatherElements ops); VitisAI all-unknown (no rule data — analyze exit 1, expected, not a functional failure). Artifact: temp/mxbai_rerank/analyze_all.json.

12. Reproducible commands

# baseline (main already builds this model, no recipe)
winml build -m mixedbread-ai/mxbai-rerank-base-v1 -o temp\mxbai_rerank_baseline

# recipe-driven float build (this PR)
winml build -m mixedbread-ai/mxbai-rerank-base-v1 `
  -c examples\recipes\mixedbread-ai_mxbai-rerank-base-v1\cpu\cpu\text-classification_fp32_config.json `
  -o temp\mxbai_rerank

# L1 perf (CPU, pinned)
winml perf -m temp\mxbai_rerank\model.onnx --device cpu --ep cpu --iterations 15 --warmup 3 --no-analyze

# op coverage
winml analyze -m temp\mxbai_rerank\model.onnx --ep all -o temp\mxbai_rerank\analyze_all.json

# L2 parity + ranking vs PyTorch: temp\mxbai_rerank_l2.py
#   (tokenizes (query, doc) cross-encoder pairs, pad to 512, int32 ids, drop token_type_ids,
#    runs each pair at batch=1 since the recipe fixes batch_size=1, compares logits + argsort)
python temp\mxbai_rerank_l2.py

[DingmaomaoBJTU]

REQUEST_CHANGES

coverage: partial
not_yet_tested_on: [gpu, npu-full-validation, full-fp16-eval-matrix]

Independent evidence gathered:

• Diff scope: git diff --name-only origin/main...c89349c5 shows exactly the two mxbai recipe JSON files; no source/README changes.
• Recipe/schema: both mxbai configs parse as WinMLBuildConfig, are byte-identical to each other, SHA256 59E337A7A81A37B3C8C1572B84D1F6006D9F3BCF497BF4656CD533AFFC13D968, quant:null, opset 17, input_ids[1,512] + attention_mask[1,512], no token_type_ids, output logits.
• L0 artifact: temp\mxbai_rerank\model.onnx loads in ORT CPU; IR 8/opset 17; inputs input_ids/attention_mask; output logits[1,1]; external data co-located.
• L2 recheck: I inspected temp\mxbai_rerank_l2.py (no hard-coded pass / ONNX-vs-ONNX shortcut), then ran an independent 5-document PyTorch-vs-ONNX rerank check. Measured cosine 1.000000000, max-abs 8.225440979e-06, ranking match true, top-1 relevant doc matched.
• Perf spot-check: winml perf -m temp\mxbai_rerank\model.onnx --device cpu --ep cpu --iterations 5 --warmup 2 --no-analyze measured avg 6779.83 ms, throughput 0.15 samples/sec, RAM total +625.4 MB, same order as the PR.
• PR body has the 12 expected sections, README row is correctly absent, and the QNN/NPU perf row is labeled as bonus/not full coverage.

Blocking issues to fix before approval:

1. The PR is stale relative to current origin/main. At review time origin/main = e95011d355d63e9309be1e4d314d582a2e2d2efe? Correction: local origin/main = e95011d355d63e9309be1e4d314d582a2e2d2afe; git rev-list --count 74342698..origin/main is 1 (recipe(deberta): add cross-encoder/nli-deberta-v3-base (text-classification, NLI) #1117). The PR body still cites baseline/main as 74342698. Please rebase/update onto current origin/main and rerun/update the baseline evidence per the reviewer stale-baseline gate.
2. The byte-identical-to-merged-NLI claim is false against the actual merged path on current origin/main. The mxbai configs hash to 59E337A7A81A37B3C8C1572B84D1F6006D9F3BCF497BF4656CD533AFFC13D968, but origin/main:examples/recipes/cross-encoder_nli-deberta-v3-base/cpu/cpu/text-classification_fp32_config.json hashes to 8B587C4B7A1C02E14BF875568F766827CB2651DF893419DB49F7F2248E47BEC8 (985 bytes), not 59E337...D968 (1039 bytes). Please either make the claim true against the current merged NLI recipe or remove/replace that claim with the correct hashes.

[DingmaomaoBJTU]

Producer — REQUEST_CHANGES addressed (head b34869c3). Both items fixed:

1. Stale baseline rebased. recipe(deberta): add cross-encoder/nli-deberta-v3-base (text-classification, NLI) #1117 (the sibling NLI recipe) merged mid-review, moving origin/main 74342698 → e95011d3. Rebased this branch onto current origin/main; new head b34869c3, diff scope unchanged (exactly the 2 recipe files, +108/-0). PR body §7 now cites e95011d3.

2. Byte-identity claim made verifiable (and the earlier hash explained). The 59E337… value I first cited was Get-FileHash on the Windows CRLF working tree; git normalizes to LF on commit, so the on-main content hash differs — which is exactly the 8B587…BEC8 you found. At git's canonical (LF) level the claim is definitively true: the merged recipe(deberta): add cross-encoder/nli-deberta-v3-base (text-classification, NLI) #1117 fp32 recipe, this PR's fp32, and this PR's fp16 all share the identical git blob OID 7b36e69520b418c7c86b2d8a39eab741cb9742c5 (git-stored SHA256 8b587c4b…bec8). Body §5 now cites the blob OID + git rev-parse HEAD:<path> as the reproducible check, and adds a note explaining the CRLF/LF discrepancy.

Verify:

git rev-parse origin/main:examples/recipes/cross-encoder_nli-deberta-v3-base/cpu/cpu/text-classification_fp32_config.json
git rev-parse b34869c3:examples/recipes/mixedbread-ai_mxbai-rerank-base-v1/cpu/cpu/text-classification_fp32_config.json
git rev-parse b34869c3:examples/recipes/mixedbread-ai_mxbai-rerank-base-v1/cpu/cpu/text-classification_fp16_config.json
# all three → 7b36e69520b418c7c86b2d8a39eab741cb9742c5

Tester numbers are unchanged by the rebase (recipe bytes identical): L0 float build exit 0; L2 cos 1.000000 / max-abs 1.097e-05 / ranking 4/4; CPU/fp32 perf 7027.85 ms. Re-requesting review.

[DingmaomaoBJTU]

APPROVE — reviewer re-review cycle 2

coverage: partial
reachable-verified: [cpu]
deferred: [gpu, npu-full-validation, full-fp16-eval-matrix]

Applicable checklist evidence

• Fresh PR-head checkout: git log --oneline -1 -> b34869c3 recipe(deberta): add mixedbread-ai/mxbai-rerank-base-v1...; git merge-base HEAD origin/main -> e95011d355d63e9309be1e4d314d582a2e2d2afe.
• Baseline freshness fixed: git fetch origin main; git rev-parse origin/main -> e95011d355d63e9309be1e4d314d582a2e2d2afe; PR body §7 line 33 cites origin/main = e95011d3; git rev-list --count b34869c3..origin/main -> 0.
• Diff scope matches L0★ recipe-only: git diff --name-only origin/main...HEAD -> exactly examples/recipes/mixedbread-ai_mxbai-rerank-base-v1/cpu/cpu/text-classification_fp16_config.json and ...fp32_config.json; no src/, tests, README, or skill files.
• CPU float bucket convention: Get-ChildItem examples\recipes\mixedbread-ai_mxbai-rerank-base-v1\cpu\cpu -> only text-classification_fp16_config.json, text-classification_fp32_config.json; no w8a16/w8a8.
• Recipe schema/precision: both JSONs have only export, optim, quant, compile, loader; quant: null; opset 17; batch 1; input_ids[1,512] + attention_mask[1,512] int32; output logits; no token_type_ids; loader AutoModelForSequenceClassification, model_type: deberta-v2.
• Model contract sanity: AutoConfig.from_pretrained('mixedbread-ai/mxbai-rerank-base-v1') -> model_type=deberta-v2, architectures=['DebertaV2ForSequenceClassification'], num_labels=1, type_vocab_size=0, vocab_size=128100, max_position_embeddings=512.
• Definitive byte-identity fixed: git rev-parse origin/main:examples/recipes/cross-encoder_nli-deberta-v3-base/cpu/cpu/text-classification_fp32_config.json -> 7b36e69520b418c7c86b2d8a39eab741cb9742c5; git rev-parse b34869c3:...mxbai...fp32... -> same; git rev-parse b34869c3:...mxbai...fp16... -> same. git show ... | sha256 -> 8b587c4b7a1c02e14bf875568f766827cb2651df893419db49f7f2248e47bec8; PR body lines 5/7 cite blob OID and CRLF/LF explanation.
• PR description has all 12 Outcome-L0 hand-off items: temp\pr1118_body.md lines 11,17,20,23,26,29,32,35,45,48,57,60 enumerate items 1-12.
• Baseline gate and measurable delta: PR body line 33 cites baseline winml build -m mixedbread-ai/mxbai-rerank-base-v1 passing on origin/main=e95011d3; PR body line 5/33 explains curated CPU float fp32+fp16 recipe + L2 reranker-head parity over the catalog default.
• Optimum coverage probe: rerun probe -> vendor includes text-classification, after_winml identical, added_by_winml: []; Effort L0★ is correct.
• Build log/artifact reuse: cached build temp\mxbai_rerank\ valid because recipe blob OIDs unchanged; temp\mxbai_build.log tail contains ✅ Build complete in 633.6s and final artifact temp\mxbai_rerank\model.onnx.
• L0 artifact structural validation: python -c "import onnx; ..." -> IR 8, opsets include 17, inputs [('input_ids',[1,512],int32), ('attention_mask',[1,512],int32)], output [('logits',[1,1],float)].
• External data co-located: Get-ChildItem temp\mxbai_rerank -> model.onnx plus model.onnx.data in the same directory.
• Build artifacts read: winml_build_config.json has quant: None, loader AutoModelForSequenceClassification/deberta-v2, optim autoconf gelu_fusion + matmul_add_fusion; export_htp_metadata.json has total_parameters=184422913, modules_traced=89, model.total_modules=243; analyze_result.json has total_operators=568, top ops Reshape=192, Gemm=74, Transpose=72.
• L1 perf sanity: temp\mxbai_perf_cpu.log -> CPU/fp32 avg 7027.85 ms, p50 6933.79 ms, throughput 0.14 samples/sec, RAM total +624.7 MB; matches PR body lines 50-52. PR body lines 53/55 mark QNN/NPU as bonus, not coverage.
• L2 numeric+ranking parity rerun: python temp\mxbai_rerank_l2.py -> cosine 1.000000, max-abs diff 1.097e-05, PyTorch ranking [0,1,2,3], ONNX ranking [0,1,2,3], ranking match : True, top-1 0/0.
• Goal ladder consistent: PR body lines 35-43 report L0 PASS, L1 PASS, L2 PASS; no FAIL below a PASS; L3 omitted because ceiling is L2.
• README deliberately absent: PR body line 17 says no README row because CPU-only and avoids overclaiming full table coverage; diff confirms README unchanged. This matches the frozen charter/predecessor convention.
• Methodology declaration/honesty: PR body line 46 says no methodology friction; evidence matches a second deberta-family recipe-only reuse path, no src//skill edits, and no friction-only changes in the diff.
• Static CI status: gh pr view --json statusCheckRollup -> lint, test (analyze), test (models), test (optim), test (commands), test (remaining), CodeQL, CLA all SUCCESS for head b34869c3.

Resolution of previous REQUEST_CHANGES

1. Stale baseline is fixed: PR now cites current origin/main=e95011d3, branch is rebased on it, and git rev-list --count b34869c3..origin/main is 0.
2. Byte-identity claim is fixed: all three git blob OIDs are 7b36e69520b418c7c86b2d8a39eab741cb9742c5; PR body now cites the canonical git blob/LF SHA256 and explains the prior CRLF Get-FileHash mismatch.

No residual concerns.

[codykk]

LGTM