Skip to content

Vulkan: classify conv2d method by weight shape in conv2d_local_wg_size - #22051

Open
msluszniak wants to merge 1 commit into
pytorch:mainfrom
msluszniak:ms/vulkan-conv2d-local-wg-method
Open

Vulkan: classify conv2d method by weight shape in conv2d_local_wg_size#22051
msluszniak wants to merge 1 commit into
pytorch:mainfrom
msluszniak:ms/vulkan-conv2d-local-wg-method

Conversation

@msluszniak

@msluszniak msluszniak commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #21942.

conv2d_local_wg_size() picked the convolution method from the shader name alone, and the condition it used matched every conv2d shader:

if (kernel_name.find("conv2d_pw") != npos ||
    (kernel_name.find("conv2d") != npos &&
     kernel_name.find("conv_transpose2d") == npos)) {
  method = Conv2dMethod::Pointwise;
} else {
  method = Conv2dMethod::SlidingWindow;
}

The sliding window shader is itself named conv2d, so it matched and was labelled Pointwise, making the SlidingWindow branch unreachable for every conv2d variant. Only conv_transpose2d reached the else.

The sibling conv2d_global_wg_size() directly above uses the identical outer name test but then disambiguates by inspecting the weight's spatial extent, so the two could disagree about the same dispatch: the global size computed as sliding window while the local size was computed as pointwise. This factors that classification into one function used by both, so they cannot drift apart again.

Why this is a restoration, not a new heuristic

The weight-shape check arrived with #13173, which introduced the tuned {64 / y, y, 1} local size for pointwise convolutions. Before that commit the dispatch used create_local_wg_size(global_size) for every method:

-  const utils::uvec3 local_size = graph.create_local_wg_size(global_size);

The name test swept sliding window convolutions into the new pointwise size along with the pointwise ones, so this restores the local size they had before #13173.

Scope of the behavior change

Only sliding window conv2d changes.

  • Pointwise was classified correctly before and after, so it keeps the tuned size.
  • Depthwise is routed to conv2d_dw_impl() before this dispatch and never reaches either function.
  • Transposed resolves to the same non-pointwise branch as before (conv_transpose2d does not contain the substring conv2d, so it already fell through to create_local_wg_size).

create_local_wg_size() is pure arithmetic on the global workgroup size, so the exact before/after can be computed on the host. Across 7452 realistic non-pointwise conv2d output shapes (spatial extents 1 to 512, 3 to 2048 output channels):

  • every case is 64 threads per group, before and after, so occupancy per dispatch is unchanged and nothing exceeds 64
  • only the group shape differs, most commonly {8, 8, 1} to one of {8, 4, 2}, {4, 8, 2}, {2, 8, 4}, {4, 2, 8}

For example, a 128x128x64 output has global size {128, 128, 16} and goes from {8, 8, 1} to {8, 4, 2}.

Since sliding window convolutions were last benchmarked under create_local_wg_size before #13173, and the pointwise tuning in that commit was not aimed at them, I would expect this to be neutral to positive. I do not have perf numbers across GPUs, so if you would like this validated on a specific device before landing, say which and I will run it.

Test plan

vulkan_backend builds clean. The change is a refactor of method classification plus the restored branch; existing conv2d correctness coverage applies unchanged, since the local workgroup size affects scheduling and not results.

cc @SS-JIA @manuelcandales @digantdesai @cbilgin

@msluszniak
msluszniak requested a review from SS-JIA as a code owner August 22, 2026 17:20
@pytorch-bot

pytorch-bot Bot commented Aug 22, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22051

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot Bot added the module: vulkan Issues related to the Vulkan delegate and code under backends/vulkan/ label Aug 22, 2026
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 22, 2026
@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Comment thread backends/vulkan/runtime/graph/ops/impl/Convolution.cpp Outdated
if (shader.kernel_name.find("conv_transpose2d") != std::string::npos) {
return Conv2dMethod::Transposed;
}
return Conv2dMethod::SlidingWindow;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, just to make sure this function covers all possible Conv2dMethod entries, would recommend detecting Conv2dMethod::Depthwise as well.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess that now every enum entry is reachable.

@SS-JIA SS-JIA left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, LGTM! Just have a few small comments. Thanks for the fix!

conv2d_local_wg_size() picked the convolution method from the shader name
alone, and the condition it used matched every conv2d shader:

    if (kernel_name.find("conv2d_pw") != npos ||
        (kernel_name.find("conv2d") != npos &&
         kernel_name.find("conv_transpose2d") == npos)) {
      method = Conv2dMethod::Pointwise;
    } else {
      method = Conv2dMethod::SlidingWindow;
    }

The sliding window shader is itself named "conv2d", so it matched and was
labelled Pointwise, which made the SlidingWindow branch unreachable for
every conv2d variant. Only conv_transpose2d reached the else.

The sibling conv2d_global_wg_size() directly above uses the identical
outer name test but then disambiguates by inspecting the weight's spatial
extent, so the two functions could disagree about the same dispatch: the
global size computed as sliding window while the local size was computed
as pointwise.

That second step arrived with pytorch#13173, which introduced the tuned
{64 / y, y, 1} local size for pointwise convolutions. Before it, conv2d
used create_local_wg_size() for every method. The name test swept sliding
window convolutions into the new pointwise size along with it, so this
restores what they had before that commit.

Factors the classification into infer_conv2d_method_from_shader(), used
by both, so they cannot drift apart again. It resolves every Conv2dMethod
entry: depthwise and transposed by their own shader names, pointwise and
sliding window by the weight's spatial extent once the names no longer
separate them. In practice only sliding window changes behavior here.
Depthwise is routed to conv2d_dw_impl() before this dispatch and never
reaches either function, pointwise was already classified correctly, and
transposed resolves to the same branch as before.

Both local sizes are always 64 threads per group; only the group shape
differs, for instance {8, 8, 1} to {8, 4, 2} for a 128x128x64 output.

Fixes pytorch#21942
@msluszniak
msluszniak force-pushed the ms/vulkan-conv2d-local-wg-method branch from c92d4ae to 9696a3f Compare August 26, 2026 20:43
@msluszniak
msluszniak requested a review from SS-JIA August 26, 2026 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. module: vulkan Issues related to the Vulkan delegate and code under backends/vulkan/

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Vulkan: conv2d_local_wg_size() classifies by shader name only, making its SlidingWindow branch unreachable

3 participants