Vulkan: classify conv2d method by weight shape in conv2d_local_wg_size - #22051
Vulkan: classify conv2d method by weight shape in conv2d_local_wg_size#22051msluszniak wants to merge 1 commit into
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22051
Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
| if (shader.kernel_name.find("conv_transpose2d") != std::string::npos) { | ||
| return Conv2dMethod::Transposed; | ||
| } | ||
| return Conv2dMethod::SlidingWindow; |
There was a problem hiding this comment.
also, just to make sure this function covers all possible Conv2dMethod entries, would recommend detecting Conv2dMethod::Depthwise as well.
There was a problem hiding this comment.
I guess that now every enum entry is reachable.
SS-JIA
left a comment
There was a problem hiding this comment.
Overall, LGTM! Just have a few small comments. Thanks for the fix!
conv2d_local_wg_size() picked the convolution method from the shader name
alone, and the condition it used matched every conv2d shader:
if (kernel_name.find("conv2d_pw") != npos ||
(kernel_name.find("conv2d") != npos &&
kernel_name.find("conv_transpose2d") == npos)) {
method = Conv2dMethod::Pointwise;
} else {
method = Conv2dMethod::SlidingWindow;
}
The sliding window shader is itself named "conv2d", so it matched and was
labelled Pointwise, which made the SlidingWindow branch unreachable for
every conv2d variant. Only conv_transpose2d reached the else.
The sibling conv2d_global_wg_size() directly above uses the identical
outer name test but then disambiguates by inspecting the weight's spatial
extent, so the two functions could disagree about the same dispatch: the
global size computed as sliding window while the local size was computed
as pointwise.
That second step arrived with pytorch#13173, which introduced the tuned
{64 / y, y, 1} local size for pointwise convolutions. Before it, conv2d
used create_local_wg_size() for every method. The name test swept sliding
window convolutions into the new pointwise size along with it, so this
restores what they had before that commit.
Factors the classification into infer_conv2d_method_from_shader(), used
by both, so they cannot drift apart again. It resolves every Conv2dMethod
entry: depthwise and transposed by their own shader names, pointwise and
sliding window by the weight's spatial extent once the names no longer
separate them. In practice only sliding window changes behavior here.
Depthwise is routed to conv2d_dw_impl() before this dispatch and never
reaches either function, pointwise was already classified correctly, and
transposed resolves to the same branch as before.
Both local sizes are always 64 threads per group; only the group shape
differs, for instance {8, 8, 1} to {8, 4, 2} for a 128x128x64 output.
Fixes pytorch#21942
c92d4ae to
9696a3f
Compare
Summary
Fixes #21942.
conv2d_local_wg_size()picked the convolution method from the shader name alone, and the condition it used matched every conv2d shader:The sliding window shader is itself named
conv2d, so it matched and was labelledPointwise, making theSlidingWindowbranch unreachable for every conv2d variant. Onlyconv_transpose2dreached theelse.The sibling
conv2d_global_wg_size()directly above uses the identical outer name test but then disambiguates by inspecting the weight's spatial extent, so the two could disagree about the same dispatch: the global size computed as sliding window while the local size was computed as pointwise. This factors that classification into one function used by both, so they cannot drift apart again.Why this is a restoration, not a new heuristic
The weight-shape check arrived with #13173, which introduced the tuned
{64 / y, y, 1}local size for pointwise convolutions. Before that commit the dispatch usedcreate_local_wg_size(global_size)for every method:The name test swept sliding window convolutions into the new pointwise size along with the pointwise ones, so this restores the local size they had before #13173.
Scope of the behavior change
Only sliding window conv2d changes.
conv2d_dw_impl()before this dispatch and never reaches either function.conv_transpose2ddoes not contain the substringconv2d, so it already fell through tocreate_local_wg_size).create_local_wg_size()is pure arithmetic on the global workgroup size, so the exact before/after can be computed on the host. Across 7452 realistic non-pointwise conv2d output shapes (spatial extents 1 to 512, 3 to 2048 output channels):{8, 8, 1}to one of{8, 4, 2},{4, 8, 2},{2, 8, 4},{4, 2, 8}For example, a 128x128x64 output has global size
{128, 128, 16}and goes from{8, 8, 1}to{8, 4, 2}.Since sliding window convolutions were last benchmarked under
create_local_wg_sizebefore #13173, and the pointwise tuning in that commit was not aimed at them, I would expect this to be neutral to positive. I do not have perf numbers across GPUs, so if you would like this validated on a specific device before landing, say which and I will run it.Test plan
vulkan_backendbuilds clean. The change is a refactor of method classification plus the restored branch; existing conv2d correctness coverage applies unchanged, since the local workgroup size affects scheduling and not results.cc @SS-JIA @manuelcandales @digantdesai @cbilgin