Add --export-type (generic | optimized) selector to winml build#1104
Add --export-type (generic | optimized) selector to winml build#1104DingmaomaoBJTU wants to merge 7 commits into
--export-type (generic | optimized) selector to winml build#1104Conversation
Replace the implicit device+ep auto-routing to the genai bundle with an explicit --export-type flag (issue #1090): - generic (default): stock single/composite ONNX build. - specialized: builds the family's registered runtime-specialized recipe (today the onnxruntime-genai NPU bundle), inferring the recipe's target. Fails fast when the family has no recipe, the input is a pre-exported .onnx, it is module mode, or --ep/--device contradicts the recipe. Recipes now declare supported_targets ((ep, device) pairs, validated at registration). The specialized path infers the first target when the user pins neither --ep nor --device, and building a bundle stays hardware-independent (no device probing). Omitting --export-type preserves the backward-compatible shortcut: a registered family with an explicit --ep qnn on an NPU target still routes to its bundle. --export-type generic always forces the stock build.
Resolve build.py conflict by keeping both the new --export-type option and #1083's dynamic export controls (--shape-config/--input-specs/ --export-config/--dynamic-axes), in the option decorators and the build() signature.
--export-type (generic | specialized) selector to winml build--export-type (generic | optimized) selector to winml build
| # model-specific value lives in the recipe. ``--export-type generic`` (or | ||
| # any other device/ep combination without the flag) keeps the stock | ||
| # single/composite build. | ||
| if _maybe_build_genai_bundle( |
There was a problem hiding this comment.
This optimized path is still reached only after the earlier if ep is None: resolve_check_device_ep(...) block has run. That means winml build -m Qwen/Qwen3-0.6B -o out --export-type optimized still probes local hardware and can fail before the recipe target is inferred, which contradicts the new contract/docs that optimized builds are hardware-independent and don't require detecting the NPU locally. Can we route explicit --export-type optimized before auto-resolving device/EP, or skip that resolution when the optimized target will be inferred from the recipe?
There was a problem hiding this comment.
Good catch — fixed in ffc3af9. --export-type optimized now skips the resolve_check_device_ep auto-EP probe entirely: the optimized branch resolves (ep, device) from the recipe via _resolve_optimized_target, so the build no longer depends on detecting the accelerator locally. Added a regression test that makes resolve_check_device_ep raise and asserts the bundle still builds (and the probe is not called).
The implicit optimized-bundle shortcut required --device to be explicitly provided (is_cli_provided), so omitting it (default `auto`) behaved differently from typing --device auto: --ep qnn alone fell through to the generic pipeline instead of the NPU bundle. Drop the --device provided-guard so the omitted `auto` default is resolved by the existing auto-resolution below, matching explicit --device auto. The --ep guard is kept (a bare --device npu can be served by non-QNN EPs, so it intentionally does not route). Add regression tests for --ep qnn without --device (routes when auto->npu, falls through when auto resolves to non-npu) and update the shortcut docstring plus build/sample docs.
|
For your description
If it is the case, I suggest we resolve first and then error that the --export-type is not supported for ep=xx, device=xx Infer target from recipe is bad due to
|
An earlier `if ep is None: resolve_check_device_ep(...)` block probed the host and picked an EP before the genai-bundle routing. That made `winml build -m <model> --export-type optimized` -- which infers its target from the recipe and is hardware-independent by contract -- still probe local hardware and fail on a machine without the recipe's accelerator (e.g. no NPU) before the recipe target was ever inferred. Skip that auto-EP resolution when --export-type optimized is explicitly requested; the optimized branch resolves (ep, device) from the recipe via _resolve_optimized_target. Add a regression test asserting the probe is not called (and the bundle still builds) even when resolve_check_device_ep would raise. Addresses review feedback on PR #1104.
Closes #1090.
Summary
Replaces the implicit device+ep auto-routing to the onnxruntime-genai bundle with an explicit
--export-typeselector onwinml build, so choosing between a stock ONNX export and a runtime-optimized bundle is a deliberate, discoverable decision instead of a side effect of--ep qnn+ NPU.--export-typegeneric(default)optimizedBehavior
optimizedinfers the target. The recipe declares itssupported_targets((ep, device)pairs), so--ep/--deviceare optional; the first target is inferred (Qwen3 →qnn/npu). Building a bundle is hardware-independent, so this path does not probe the local device.optimizederrors when the family has no recipe, the input is a pre-exported.onnx, it is module mode, or an explicitly pinned--ep/--devicecontradicts the recipe.--export-typeis omitted, the existing shortcut still applies: a registered family with an explicit--ep qnnon an NPU target (--device npu, or--device autoresolving to the NPU) routes to its bundle.--export-type genericalways forces the stock build, even on the NPU.Changes
genai_bundle.py): newGenaiTargetfrozen dataclass and a requiredsupported_targetsfield onGenaiBundleRecipe, validated (non-empty) at registration. New symbols exported from the package__init__.supported_targets=(GenaiTarget(ep="qnn", device="npu"),).winml build(build.py): adds the--export-typechoice option and refactors the routing into small helpers (_resolve_genai_recipe,_resolve_optimized_target,_maybe_build_genai_bundle) that keep the switch data-driven and architecture-agnostic.docs/commands/build.md(flag row + reframed genai-bundle section) anddocs/samples/qwen3-genai-bundle.mdnow lead with--export-type optimizedand document the shortcut as backward-compatible.Sample
Build the optimized onnxruntime-genai NPU bundle for a decoder LLM (the target is inferred from the recipe, so
--ep/--deviceare optional):Force a stock ONNX build instead:
The selector is discoverable in
--help:Fail-fast validation when the model type has no optimized recipe (verified in terminal):
Tests
New CLI routing tests (infer-and-build,
genericforces stock, unregistered-family error) plus focused unit tests for target inference/contradiction and theoptimizedpreconditions; registry tests for the empty-supported_targetsguard. The genai build/routing suites (tests/unit/commands/test_build_genai_bundle.pyandtests/unit/models/winml/) pass;ruff checkand strictmypyclean.