Skip to content

recipe(google/owlv2-base-patch16-ensemble): add zero-shot-object-detection fp16, w8a8 and w8a16 recipes#1088

Open
codykk wants to merge 1 commit into
mainfrom
yongyue/add-owlv2-base-patch16-ensemble
Open

recipe(google/owlv2-base-patch16-ensemble): add zero-shot-object-detection fp16, w8a8 and w8a16 recipes#1088
codykk wants to merge 1 commit into
mainfrom
yongyue/add-owlv2-base-patch16-ensemble

Conversation

@codykk

@codykk codykk commented Jul 10, 2026

Copy link
Copy Markdown

Add fp16, w8a8, and w8a16 quantized recipes for google/owlv2-base-patch16-ensemble (zero-shot object detection, 155M params). Effort L0★ — first zero-shot-object-detection recipe in the repo, no source edits. Goal ceiling L1 reached: all three recipes build and the w8a16 variant perfs on CPU. Quantization delivers up to 3.9x model size reduction (585.6 MB fp32 → 148.4 MB w8a8).


12-Item Verification

1. Recipe path(s)

  • examples/recipes/google_owlv2-base-patch16-ensemble/zero-shot-object-detection_fp16_config.json
  • examples/recipes/google_owlv2-base-patch16-ensemble/zero-shot-object-detection_w8a16_config.json
  • examples/recipes/google_owlv2-base-patch16-ensemble/zero-shot-object-detection_w8a8_config.json

2. README row

Added to examples/recipes/README.md:

| google/owlv2-base-patch16-ensemble | zero-shot-object-detection |

3. Build output dir

temp/owlv2_fp16/   (fp16)
temp/owlv2_build/  (w8a16)
temp/owlv2_w8a8/   (w8a8)

4. Build log

Recipe Status Time Artifact Size
fp16 ✅ Build complete 59.3s 585.6 MB
w8a16 ✅ Build complete 119.7s 175.2 MB
w8a8 ✅ Build complete 125.4s 148.4 MB

All artifacts: 1423 ONNX nodes, opset 17, IR version 8.

5. Appended findings

N/A — no model_knowledge/owlv2.json existed prior; no methodology friction observed that warrants a new finding. The model exported cleanly via the existing vendor OwlV2OnnxConfig registered by winml.

6. Optimum-coverage probe

{'vendor': [], 'after_winml': ['feature-extraction', 'zero-shot-object-detection'], 'added_by_winml': ['feature-extraction', 'zero-shot-object-detection']}
  • Verdict: WINML-ONLY — no vendor Optimum coverage; winml registers OwlV2OnnxConfig for both tasks.
  • Cross-check: model_type = "owlv2", architectures = ["Owlv2ForObjectDetection"], is_encoder_decoder = False.

7. Claimed (Effort, Goal, Outcome)

Axis Tier Justification
Effort L0★ First zero-shot-object-detection recipe; no source edits; OnnxConfig already registered by winml
Goal L1 (ceiling) Build + perf verified; baseline already passes on main so L0 is inherited
Outcome L0 Recipe JSON × 3 + README row

8. Goal-ladder verdict table

Tier Verdict Evidence
L0 (build) PASS All three recipes (fp16/w8a16/w8a8) build successfully; ✅ Build complete in 59.3s / 119.7s / 125.4s
L1 (perf) PASS CPU w8a16: mean 3865.88ms, P50 3859.67ms, 0.26 samples/s, RAM +2836.8 MB
L2 (numeric) N/A Not in ceiling
L3 (eval) N/A Not in ceiling

Baseline build on main (defa3f63): PASS — L0 is inherited from baseline; the contribution's delta is quantization.

9. Methodology-evolution declaration

No methodology friction observed. No new _meta-NNN finding needed.

10. Perf & eval data

EP / Device Precision Verdict Mean P50 Throughput RAM Δ VRAM Δ
CPUExecutionProvider / cpu w8a16 PASS 3865.88 ms 3859.67 ms 0.26 samples/s +2836.8 MB

Available providers on test host: ['DmlExecutionProvider', 'CPUExecutionProvider']

DmlExecutionProvider was not in charter.target_eps (CPU-only target), so no HOST-BLOCKED row.

11. Component / op-level data

N/A — winml analyze runtime check rules not populated on this host (source-tree build, no parquet rules downloaded). The build-embedded analyze_result.json only covers CPU (no rules → all ops unknown). See _meta-013.

Note from build: Einsum op in /class_head/Einsum flagged as unsupported during optimize (3 occurrences), but optimization and quantization completed successfully around it.

12. Reproducible commands

# Environment
winml --version
git rev-parse HEAD  # defa3f63ffdc5c4cd3c252a4b3f60fcc9939f953 (origin/main)
python -c "import onnxruntime as ort; print(ort.get_available_providers())"
# ['DmlExecutionProvider', 'CPUExecutionProvider']

# Baseline (no recipe, main branch)
winml build -m google/owlv2-base-patch16-ensemble -o temp/baseline_owlv2/ --ep cpu --device cpu --no-analyze --no-optimize --no-quant --no-compile --rebuild

# Build fp16
winml build -c examples/recipes/google_owlv2-base-patch16-ensemble/zero-shot-object-detection_fp16_config.json -m google/owlv2-base-patch16-ensemble -o temp/owlv2_fp16/ --rebuild

# Build w8a16
winml build -c examples/recipes/google_owlv2-base-patch16-ensemble/zero-shot-object-detection_w8a16_config.json -m google/owlv2-base-patch16-ensemble -o temp/owlv2_build/ --rebuild

# Build w8a8
winml build -c examples/recipes/google_owlv2-base-patch16-ensemble/zero-shot-object-detection_w8a8_config.json -m google/owlv2-base-patch16-ensemble -o temp/owlv2_w8a8/ --rebuild

# Perf (w8a16 on CPU)
winml perf -m temp/owlv2_build/model.onnx --device cpu --ep cpu

@codykk codykk changed the title Add recipe: google/owlv2-base-patch16-ensemble (zero-shot-object-dete… recipe(google/owlv2-base-patch16-ensemble): add zero-shot-object-detection fp16, w8a8 and w8a16 recipes Jul 10, 2026
@codykk codykk marked this pull request as ready for review July 10, 2026 07:52
@codykk codykk requested a review from a team as a code owner July 10, 2026 07:52
@codykk codykk force-pushed the yongyue/add-owlv2-base-patch16-ensemble branch from aed2103 to fef0bc5 Compare July 13, 2026 01:29
…ction)

Add fp16, w8a8, and w8a16 quantized recipes for OWLv2 base model,
enabling zero-shot object detection on WinML.

Build verified:
- fp16:  PASS (585.6 MB, 59.3s)
- w8a8:  PASS (148.4 MB, 125.4s)
- w8a16: PASS (175.2 MB, 119.7s)

Perf verified (w8a16, CPU): mean 3865.88ms, 0.26 samples/s
@codykk codykk force-pushed the yongyue/add-owlv2-base-patch16-ensemble branch from fef0bc5 to 2bf8276 Compare July 15, 2026 07:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant