recipe(cross-encoder/ms-marco-MiniLM-L6-v2): add text-classification fp16 and w8a16 recipes

[codykk]

Add recipe configurations for cross-encoder/ms-marco-MiniLM-L6-v2 (BertForSequenceClassification, 22.7M params, 6 layers) with fp16 and w8a16 quantized variants. The w8a16 recipe delivers 61% model size reduction (86.7 MB → 33.4 MB). Effort L0, Goal L1 PASS on CPU, Outcome L0.

---

1. Recipe path(s)

• examples/recipes/cross-encoder_ms-marco-MiniLM-L6-v2/text-classification_fp16_config.json
• examples/recipes/cross-encoder_ms-marco-MiniLM-L6-v2/text-classification_w8a16_config.json

2. README row

Added cross-encoder/ms-marco-MiniLM-L6-v2 | text-classification to examples/recipes/README.md.

3. Build output dir

• fp16: temp/verify_cross-encoder_ms-marco-MiniLM-L6-v2_fp16/
• w8a16: temp/verify_cross-encoder_ms-marco-MiniLM-L6-v2_w8a16/

4. Build log

• fp16: ✅ Build complete in 23.6s (export 4.8s, optimize 18.2s)
• w8a16: ✅ Build complete in 33.7s (export 4.9s, optimize 19.2s, quantize 8.6s)

Structural validation (w8a16): IR version 8, inputs input_ids [1, 512], attention_mask [1, 512], token_type_ids [1, 512], output logits [1, 1].

5. Appended findings

No existing model_knowledge/bert.json in skill repo. No methodology friction observed — standard BERT text-classification, fully vendor-covered (Optimum + winml defaults). Note: despite the HuggingFace text-classification task label, this is a cross-encoder ranking / regression model (MS MARCO passage ranking). Output is logits [1, 1] — a single relevance score, not a multi-class softmax. Standard text-classification eval datasets (MRPC, SST-2) are incompatible due to label mismatch (KeyError: 'equivalent'). Eval section intentionally omitted from recipe.

6. Optimum-coverage probe

{"vendor": ["feature-extraction", "fill-mask", "multiple-choice", "question-answering", "text-classification", "token-classification"],
 "after_winml": ["feature-extraction", "fill-mask", "multiple-choice", "question-answering", "text-classification", "token-classification"],
 "added_by_winml": [],
 "verdict": "VENDOR-ONLY"}

7. Claimed (Effort, Goal, Outcome)

Axis	Tier
Effort	L0 (recipe only, no source edits)
Goal ceiling	L1 (build + perf)
Outcome	L0 (recipe + README row + PR report)

8. Goal-ladder verdict table

Tier	Verdict	Evidence
L0	PASS	fp16: ✅ Build complete in 23.6s, artifact 86.7 MB; w8a16: ✅ Build complete in 33.7s, artifact 33.4 MB. onnx.load OK, IR=8, inputs match recipe.
L1	PASS	CPU perf below. fp16 23.14 ms avg; w8a16 59.71 ms avg.

9. Methodology-evolution declaration

No methodology friction observed. Standard BERT text-classification path, fully covered by Optimum vendor defaults. One finding: cross-encoder ranking models (output logits [1, 1]) are labeled text-classification on HuggingFace but incompatible with standard classification eval datasets — winml eval with MRPC fails with KeyError: 'equivalent' at label alignment. Eval section removed from recipe; L3 goal not applicable for this model type without a ranking-specific eval dataset.

10. Perf & eval data

EP / Device	Precision	Verdict	Mean	p50	Throughput	RAM Δ
CPUExecutionProvider / cpu	fp32 (fp16 recipe)	PASS	23.14 ms	22.85 ms	43.21 samples/s	+182.7 MB
CPUExecutionProvider / cpu	w8a16	PASS	59.71 ms	58.54 ms	16.75 samples/s	+171.6 MB

Size delta: baseline 86.7 MB → w8a16 33.4 MB (61% reduction).

Note: w8a16 latency is higher than fp32 due to QDQ overhead on CPU — known for small models where quantization/dequantization cost exceeds compute savings. The value is in model size reduction for deployment-constrained scenarios.

11. Component / op-level data

Runtime check rules not available on this host (source-tree build, no parquet rules downloaded). winml analyze skipped — to be populated when rules are available.

12. Reproducible commands

# Baseline (no recipe, main)
git rev-parse origin/main  # 670ad35e56b8e416bacc8ca48b13289b87c0a5f7
winml build -m cross-encoder/ms-marco-MiniLM-L6-v2 -o temp/baseline_ms-marco-MiniLM-L6-v2 --ep cpu --device cpu --no-analyze --no-optimize --no-quant --no-compile --rebuild

# Build fp16
winml build -c examples/recipes/cross-encoder_ms-marco-MiniLM-L6-v2/text-classification_fp16_config.json -m cross-encoder/ms-marco-MiniLM-L6-v2 -o temp/verify_cross-encoder_ms-marco-MiniLM-L6-v2_fp16/ --rebuild

# Build w8a16
winml build -c examples/recipes/cross-encoder_ms-marco-MiniLM-L6-v2/text-classification_w8a16_config.json -m cross-encoder/ms-marco-MiniLM-L6-v2 -o temp/verify_cross-encoder_ms-marco-MiniLM-L6-v2_w8a16/ --rebuild

# Perf fp16
winml perf -m temp/verify_cross-encoder_ms-marco-MiniLM-L6-v2_fp16/model.onnx --device cpu --ep cpu

# Perf w8a16
winml perf -m temp/verify_cross-encoder_ms-marco-MiniLM-L6-v2_w8a16/model.onnx --device cpu --ep cpu

# Structural validation
python -c "import onnx; m=onnx.load('temp/verify_cross-encoder_ms-marco-MiniLM-L6-v2_w8a16/model.onnx', load_external_data=False); print(m.ir_version, [(i.name, [d.dim_value or d.dim_param for d in i.type.tensor_type.shape.dim]) for i in m.graph.input])"

[codykk]

@microsoft-github-policy-service agree company="Microsoft"