Add vitpose-plus-huge recipe + fix FP16 2GB protobuf limit#1099
Draft
ssss141414 wants to merge 2 commits into
Draft
Add vitpose-plus-huge recipe + fix FP16 2GB protobuf limit#1099ssss141414 wants to merge 2 commits into
ssss141414 wants to merge 2 commits into
Conversation
Add keypoint-detection fp16 recipe for usyd-community/vitpose-plus-huge. The ViT-Huge fp32 export is ~3.4 GB, and ORT's convert_float_to_float16 runs shape inference by default, which serializes the proto and exceeds protobuf's 2 GB message limit (EncodeError). convert_to_fp16 now auto-detects large models (>=90% of 2 GB via ByteSize with a raise-fallback) and passes disable_shape_infer=True, unblocking all >2 GB models. Validated CPU-only: genuine fp16 (FLOAT16=948, FLOAT32=0), build 661.9s, perf P50 746.49ms.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds winml-cli support for
usyd-community/vitpose-plus-huge(keypoint-detection) and fixes an FP16-conversion crash that blocks every model whose fp32 export exceeds the protobuf 2 GB message limit.Changes
examples/recipes/usyd-community_vitpose-plus-huge/keypoint-detection_fp16_config.json— new fp16 recipe (inputpixel_values [1,3,256,192]float32, outputheatmaps,VitPoseForPoseEstimation/vitposeloader, COCO-keypoints eval block). Mirrors the existingvitpose-plus-{base,small,large}recipes.src/winml/modelkit/quant/fp16.py—convert_to_fp16now auto-detects large models and disables shape inference during conversion.Why the FP16 fix
The ViT-Huge fp32 export is ~3.4 GB. ORT's
convert_float_to_float16runs ONNX shape inference by default, which serializes the model proto and exceeds protobuf's hard 2 GB message limit, raisinggoogle.protobuf.message.EncodeError: Failed to serialize proto. The fix estimates model size viaModelProto.ByteSize()(which itself raises past 2 GB — treated as a definitive "too large" signal) and passesdisable_shape_infer=Truewhen the model is at or above 90% of the 2 GB limit. This unblocks all >2 GB models, not just this one.Validation
CPU-only (no NPU/GPU hardware available in this environment):
Build complete in 661.9s, FP16 stage 71.4s (previously crashed withEncodeError).model.onnx.data1.8 GB (half of the 3.6 GB fp32).winml perf --device cpu --ep cpu: P50 746.49 ms, throughput 1.35 samples/sec.Per-EP (QNN/OpenVINO/VitisAI/TensorRT) accuracy eval on all 10 buckets is deferred to CI on the appropriate hardware. This model is therefore not added to the recipe README total (which certifies all-10-bucket passes), consistent with the existing sibling vitpose recipes.
Draft — opening for review of the FP16 large-model fix and the recipe.