Skip to content

Add an explicit --export-type (GENERIC | SPECIALIZED) selector for winml build instead of device+ep auto-routing #1090

Description

@DingmaomaoBJTU

Context

PR #1081 makes winml build emit an onnxruntime-genai bundle for a registered
decoder-LLM family (currently Qwen3). The trigger is intentionally implicit: a registered
family plus an explicit --ep qnn plus an NPU target (--device npu, or a
--device auto that resolves to the NPU). Every other target keeps the stock
single/composite build.

So a single family already has two possible outputs, selected today only by the
device+ep pair:

  • a single-model / composite ONNX build (today's stock output), and
  • a runtime-specialized multi-component bundle — the onnxruntime-genai layout
    (ctx / iter / embeddings / lm_head + genai_config.json + tokenizer), produced
    from a registered recipe.

Note both outputs are already target-aware (the stock build does device-aware quantization
and can compile an EPContext for a specific EP). The real distinction is output topology
and consuming runtime
— one ONNX model for ORT single-shot inference vs. a bundle for the
onnxruntime-genai generation loop — not "portable vs target-specific."

Suggestion (from review of #1081)

@xieofxie suggested (review threads on src/winml/modelkit/commands/build.py:
3555879362,
3555913471)
that instead of auto-routing on device+ep, we expose an explicit selector for the
kind of output, rather than hiding one output behind a device/ep combination:

So in the future if we have more recipes for each, user could easily switch instead of
automatically switch and hide another.

Proposal: an explicit --export-type selector

Add an explicit export-type selector on winml build:

--export-type GENERIC | SPECIALIZED
  • GENERIC (default) — the stock single-model / composite ONNX build. This is today's
    output for every target, so the default preserves current behavior.
  • SPECIALIZED — a runtime-specialized export produced from a registered recipe (today:
    the onnxruntime-genai NPU bundle).

--device / --ep / --precision remain target descriptors within the chosen
export-type; they no longer silently decide which kind of artifact is produced.

The point: an explicit selector scales better than a device+ep pair once a family has more
than one output, and gives users the choice without hidden routing.

Naming — open. --export-type overloads "export": there is already a winml export
command and an export stage inside the build pipeline, so --export-type may read as
"the export stage's type." Alternatives to weigh: --output-kind, --artifact,
--profile, --build-type, or a registry-keyed --recipe (see Binary vs. extensible
below). GENERIC/SPECIALIZED value casing / case-insensitivity is an implementation
detail.

Key decision: does SPECIALIZED infer the target, or require it?

The recipe already knows the target it needs (today: QNN NPU). Two options:

  1. Infer (recommended). winml build -m <model> -o out --export-type SPECIALIZED
    selects the recipe's target automatically. --device/--ep are only needed to pick
    among a recipe's supported targets, and an explicit contradiction
    (e.g. --device cpu) or unavailable hardware is an error. This makes the explicit
    selector genuinely more ergonomic than the device+ep trigger it replaces.
  2. Require. The user must still pass the recipe's --device/--ep (e.g. --ep qnn
    on an NPU); a mismatch is an error. Closest to today's behavior but offers little over
    the implicit trigger.

This is the central UX decision and is currently unspecified. --device auto must be
defined under either option: it should resolve to the recipe's supported target if one is
available on the host, else error.

Validation: SPECIALIZED with an unmet target fails fast (hard error)

When --export-type SPECIALIZED is passed but the target cannot serve the selected recipe
— today the only recipe is the QNN NPU bundle, so an explicit non-QNN --ep, a
non-NPU --device, or an NPU that is unavailable on the host — the build must abort with
a clear error and a non-zero exit
.

  • This is a fail-fast error (a UsageError, like the existing -c/--use-cache
    rejections), not a soft warning that continues.
  • It must not silently fall back to GENERIC or build a mismatched artifact.

The same fail-fast applies when the family has no specialized recipe at all. Because the
user asked for SPECIALIZED explicitly, an unmet target is a user error to surface, not
something to auto-correct.

Prerequisite: recipes must declare their supported targets

The validation above is not expressible today: GenaiBundleRecipe
(models/winml/genai_bundle.py) has no ep/device field — QNN+NPU is hardcoded at the call
site (build_genai_bundle(..., ep="qnn", device="npu")) and as orchestrator defaults. This
work must first add a declarative supported-targets field to the recipe (e.g.
supported_targets: tuple[(ep, device), ...]) so build can validate --export-type SPECIALIZED against it and drive target inference, keeping the routing architecture-agnostic.

Inherited constraints under SPECIALIZED

SPECIALIZED keeps the recipe-driven pipeline, so the controls the fast path already
rejects still apply and must error consistently: -c/--config, --use-cache,
--quant/--no-quant, --optimize/--no-optimize, --analyze/--no-analyze,
--compile/--no-compile, --max-optim-iterations, --allow-unsupported-nodes.
--precision remains the one supported transformer knob.

Binary vs. extensible axis

GENERIC/SPECIALIZED is a binary, but the stated motivation is scaling to multiple
recipes per family. Two specialized recipes for one family would make SPECIALIZED
ambiguous again — the same problem, one level up. Decide whether the axis is a fixed 2-value
enum plus a later recipe sub-selector, or an extensible, registry-keyed value from the
start (e.g. --export-type <recipe-name> / a dedicated --recipe), so the primary flag
names the concrete recipe when a family has more than one.

Interaction with the current device+ep trigger

--export-type is the explicit selector and overrides routing when passed. When
omitted, the existing --ep qnn + NPU trigger from #1081 is kept as a convenience
shortcut
for SPECIALIZED, so today's opt-in, non-destructive behavior is unchanged and
no existing command breaks. --export-type GENERIC forces the stock build even for a
registered family on the NPU.

Reconcile with perf (two vocabularies)

perf already selects the engine with --runtime winml | winml-genai
(commands/perf.py). Under this proposal a user builds --export-type SPECIALIZED then
benchmarks --runtime winml-genai — two different vocabularies for the same bundle.
Options: (a) share one vocabulary across build/perf/export; (b) keep --export-type
(build artifact) and --runtime (inference engine) as deliberately distinct axes and
document the mapping; or (c) have perf auto-detect a genai bundle from its directory
layout so --runtime is rarely needed. This is a decision, not just a note — the original
issue proposed winml | winml-genai precisely for this symmetry.

Discoverability

There is no way today to learn that a model supports SPECIALIZED. Consider surfacing the
supported export-types (and their targets) in winml inspect / analyze, and/or having the
GENERIC build emit a hint when the family also has a specialized recipe.

Why it's not in #1081

The device+ep trigger was locked with the product owner for that PR: it is opt-in
(--ep qnn must be explicit), non-destructive (every other target is unchanged), and needs
no new surface. Adding an explicit --export-type flag is a broader CLI-surface design
change, so it's better evaluated on its own.

Proposed follow-up

  • Add --export-type GENERIC | SPECIALIZED as the explicit output selector for build,
    defaulting to GENERIC; settle the flag name (see Naming).
  • Decide infer vs. require for SPECIALIZED targets, including --device auto.
  • Add a declarative supported-targets field to GenaiBundleRecipe so target validation
    and inference are data-driven (prerequisite).
  • Fail-fast (non-zero exit, UsageError) when SPECIALIZED can't serve the target or the
    family has no recipe — never silently fall back to GENERIC.
  • Keep the --ep qnn + NPU trigger as a shortcut for SPECIALIZED when --export-type is
    omitted; --export-type overrides it when passed.
  • Preserve the existing SPECIALIZED control rejections (config/cache/quant/optimize/etc.).
  • Decide binary enum vs. registry-keyed recipe value for multi-recipe families.
  • Reconcile with perf's --runtime winml | winml-genai (shared vocab, documented mapping,
    or perf auto-detect).
  • Surface supported export-types for discoverability (inspect/analyze, or a build hint).

Tracking the design discussion here so #1081 can merge with the current implicit trigger.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions