[ExecuTorch][WebGPU] Test coverage for the f16 KV cache by JCNTH · Pull Request #20773 · pytorch/executorch

JCNTH · 2026-07-07T18:02:30Z

Stack from ghstack (oldest at bottom):

Tests for the opt-in f16 KV cache (stacked op diff). When built with EXECUTORCH_WEBGPU_KV_F16 and run on a shader-f16 device, the SDPA op stores the K/V cache as f16, so the entire existing sdpa_with_kv_cache golden suite (config sweep + replay + dynamic decode) exercises the f16-KV path with no new goldens needed; this diff loosens the SDPA numeric tolerance to the f16 read-precision floor when — and only when — that path is active.

Key changes:

test_webgpu_native.cpp sdpa_within_tol — under #ifdef WGPU_BACKEND_KV_F16, compare at abs 2e-3 / rel 1e-2 when the device negotiated shader-f16, else keep the strict f32 abs 1e-4 / rel 1e-3. Every SDPA config/replay/decode golden then validates the f16-KV output through the existing exemplars.

Constraints: the default (flag-OFF) test build is unchanged; on a non-shader-f16 device (including the CI software adapter) the f16 KV path stays inactive and the strict f32 tolerance applies, so there is no CI behavior change. The f16-KV numeric validation is therefore shader-f16-device-only (Canary/Metal), matching the op's opt-in gating; no CI script change (mirrors the steel-f16 tests).

Co-authored-with: Claude Code.
@exported-using-ghexport

Differential Revision: D110919973

[ghstack-poisoned]

pytorch-bot · 2026-07-07T18:02:34Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20773

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 3 Pending, 1 Unrelated Failure

As of commit 20d49dc with merge base f4b01a8 ():

NEW FAILURES - The following jobs have failed:

pull / android / build-android (gh)
pull / test-binary-size-linux-gcc / linux-job (gh)
RuntimeError: Command docker exec -t 8abf0c83b75de4ca39ecb6be612ca18c8e53860c388b58b8b21449100f0a3b4a /exec failed with exit code 1
pull / unittest / linux / linux-job (gh)
RuntimeError: Command docker exec -t 5747d7ef9445eff76fcd459ac0a20c29c76d3de65a61870f6d703e8d166e6bcf /exec failed with exit code 1
pull / unittest-editable / linux / linux-job (gh)
RuntimeError: Command docker exec -t 32532aee31036d6ab19c449a03a46677e1aec26ff86177761c514e67406b9b59 /exec failed with exit code 1

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / unittest-nxp-neutron / linux-job (gh) (trunk failure)
##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions · 2026-07-07T18:03:48Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[ghstack-poisoned]

Pull Request resolved: #20773 Tests for the opt-in f16 KV cache (stacked op diff). When built with `EXECUTORCH_WEBGPU_KV_F16` and run on a shader-f16 device, the SDPA op stores the K/V cache as f16, so the entire existing `sdpa_with_kv_cache` golden suite (config sweep + replay + dynamic decode) exercises the f16-KV path with no new goldens needed; this diff loosens the SDPA numeric tolerance to the f16 read-precision floor when — and only when — that path is active. Key changes: - `test_webgpu_native.cpp` `sdpa_within_tol` — under `#ifdef WGPU_BACKEND_KV_F16`, compare at abs `2e-3` / rel `1e-2` when the device negotiated shader-f16, else keep the strict f32 abs `1e-4` / rel `1e-3`. Every SDPA config/replay/decode golden then validates the f16-KV output through the existing exemplars. Constraints: the default (flag-OFF) test build is unchanged; on a non-shader-f16 device (including the CI software adapter) the f16 KV path stays inactive and the strict f32 tolerance applies, so there is no CI behavior change. The f16-KV numeric validation is therefore shader-f16-device-only (Canary/Metal), matching the op's opt-in gating; no CI script change (mirrors the steel-f16 tests). Co-authored-with: Claude Code. ghstack-source-id: 400708673 @exported-using-ghexport Differential Revision: [D110919973](https://our.internmc.facebook.com/intern/diff/D110919973/)

[ghstack-poisoned]

SS-JIA

Review automatically exported from Phabricator review in Meta.

[ghstack-poisoned]

SS-JIA

Review automatically exported from Phabricator review in Meta.

[ghstack-poisoned]

Pull Request resolved: #20773 Tests for the opt-in f16 KV cache (stacked op diff). When built with `EXECUTORCH_WEBGPU_KV_F16` and run on a shader-f16 device, the SDPA op stores the K/V cache as f16, so the entire existing `sdpa_with_kv_cache` golden suite (config sweep + replay + dynamic decode) exercises the f16-KV path with no new goldens needed; this diff loosens the SDPA numeric tolerance to the f16 read-precision floor when — and only when — that path is active. Key changes: - `test_webgpu_native.cpp` `sdpa_within_tol` — under `#ifdef WGPU_BACKEND_KV_F16`, compare at abs `2e-3` / rel `1e-2` when the device negotiated shader-f16, else keep the strict f32 abs `1e-4` / rel `1e-3`. Every SDPA config/replay/decode golden then validates the f16-KV output through the existing exemplars. Constraints: the default (flag-OFF) test build is unchanged; on a non-shader-f16 device (including the CI software adapter) the f16 KV path stays inactive and the strict f32 tolerance applies, so there is no CI behavior change. The f16-KV numeric validation is therefore shader-f16-device-only (Canary/Metal), matching the op's opt-in gating; no CI script change (mirrors the steel-f16 tests). Co-authored-with: Claude Code. ghstack-source-id: 401515179 @exported-using-ghexport Differential Revision: [D110919973](https://our.internmc.facebook.com/intern/diff/D110919973/)

Update

dd4538b

[ghstack-poisoned]

JCNTH had a problem deploying to cadence July 7, 2026 18:02 — with GitHub Actions Error

JCNTH temporarily deployed to cadence July 7, 2026 18:02 — with GitHub Actions Inactive

meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 7, 2026

Update

1479030

[ghstack-poisoned]

meta-codesync Bot added the meta-exported label Jul 7, 2026

JCNTH temporarily deployed to cadence July 7, 2026 18:30 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 7, 2026 18:55 — with GitHub Actions Inactive

Update

e5b9cff

[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 9, 2026 03:45 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 9, 2026 04:14 — with GitHub Actions Inactive

SS-JIA requested changes Jul 9, 2026

View reviewed changes

Update

7898e55

[ghstack-poisoned]

JCNTH had a problem deploying to cadence July 9, 2026 17:41 — with GitHub Actions Error

Update

489f9e1

[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 9, 2026 17:54 — with GitHub Actions Inactive

JCNTH had a problem deploying to cadence July 9, 2026 18:24 — with GitHub Actions Error

SS-JIA approved these changes Jul 9, 2026

View reviewed changes

Update

1f6038e

[ghstack-poisoned]

JCNTH had a problem deploying to cadence July 9, 2026 18:36 — with GitHub Actions Error

Update

20d49dc

[ghstack-poisoned]

JCNTH temporarily deployed to cadence July 9, 2026 18:59 — with GitHub Actions Inactive

JCNTH temporarily deployed to cadence July 9, 2026 19:27 — with GitHub Actions Inactive

meta-codesync Bot merged commit ddf78e4 into gh/JCNTH/11/base Jul 9, 2026
177 of 183 checks passed

meta-codesync Bot deleted the gh/JCNTH/11/head branch July 9, 2026 21:40

meta-codesync Bot temporarily deployed to cherry-pick-bot July 9, 2026 21:40 Inactive

pytorchbot mentioned this pull request Jul 9, 2026

[ExecuTorch][WebGPU] Test coverage for the f16 KV cache #20820

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[ExecuTorch][WebGPU] Test coverage for the f16 KV cache#20773

[ExecuTorch][WebGPU] Test coverage for the f16 KV cache#20773
meta-codesync[bot] merged 7 commits into
gh/JCNTH/11/basefrom
gh/JCNTH/11/head

JCNTH commented Jul 7, 2026 •

edited

Loading

Uh oh!

pytorch-bot Bot commented Jul 7, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 7, 2026

Uh oh!

SS-JIA left a comment

Uh oh!

SS-JIA left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

JCNTH commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot Bot commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20773

❌ 4 New Failures, 3 Pending, 1 Unrelated Failure

Uh oh!

github-actions Bot commented Jul 7, 2026

This PR needs a release notes: label

Uh oh!

SS-JIA left a comment

Choose a reason for hiding this comment

Uh oh!

SS-JIA left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

JCNTH commented Jul 7, 2026 •

edited

Loading

pytorch-bot Bot commented Jul 7, 2026 •

edited

Loading

This PR needs a `release notes:` label