Skip to content

[CI] Publish PyTorch 2.11 CUDA 13 release wheels - #3311

Open
bvolpato wants to merge 2 commits into
NVIDIA:mainfrom
bvolpato:bvolpato/publish-gb200-aarch64-wheels
Open

bvolpato wants to merge 2 commits into
NVIDIA:mainfrom
bvolpato:bvolpato/publish-gb200-aarch64-wheels

Conversation

@bvolpato

@bvolpato bvolpato commented Aug 4, 2026 •

Copy link
Copy Markdown

Description

Publish PyTorch 2.11/CUDA 13 release wheels for Linux x86_64 and aarch64 so matching installs can download the extension instead of compiling it locally. Keep the existing PyTorch 2.8/CUDA 12.9 builds.

Skip disk pooling when /mnt shares the root filesystem to fix the ARM release checkout failure.

Fixes #3310

Type of change

  • Documentation change (change only to the documentation, either a fix or a new content)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Infra/Build change
  • Code refactoring

Changes

  • Add Python 3.12, PyTorch 2.11/CUDA 13 builds for both architectures.
  • Include SM100 and install the CUDA development packages needed for the build.
  • Preserve checkout space when /mnt shares the root filesystem.
  • Keep compiler output visible and fail the job when the build fails or times out.
  • Check wheel tags before upload and run install/import smoke checks for the new builds.

Validation

Repository hooks and shell checks passed; actionlint reported only existing workflow diagnostics. Rerun the hooks with pre-commit run --all-files.

Wheel builds and driverless imports passed on x86_64 (PyTorch 2.11/CUDA 13, v2.18) and native aarch64 (CUDA 12.9 and CUDA 13, v2.20.2), using this PR's build scripts. GPU execution has not been tested. The release workflow does not run on PRs.

Checklist:

  • I have read and followed the contributing guidelines
  • The functionality is complete
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

@github-actions github-actions Bot added the community-contribution PRs from external contributor outside the core maintainers, representing community-driven work. label Aug 4, 2026
@greptile-apps

greptile-apps Bot commented Aug 4, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[High impact] The PR appears safe to merge; no new actionable issues were found.

Summary

Adds Python 3.12 wheels for PyTorch 2.11/CUDA 13 on Linux x86_64 and aarch64 while retaining the PyTorch 2.8/CUDA 12.9 builds.

  • Release publishing adds PyTorch 2.11 CUDA 13 wheels for both Linux architectures.
  • Build runners keep checkout space when /mnt shares the root filesystem.
  • Wheel builds show full logs and return failed build status.

No new actionable issues were found.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Release published] --> B[Expand build matrix]
  B --> C[PyTorch 2.8 / CUDA 12.9<br/>x86_64 and aarch64]
  B --> D[PyTorch 2.11 / CUDA 13<br/>x86_64 and aarch64]
  C --> E[Build and check wheel tags]
  D --> E
  E --> F{Build and checks pass?}
  F -->|No| G[Stop without upload]
  F -->|Yes| H[Install/import check for new builds]
  H --> I[Upload release asset]
Loading

Reviews (5) · Last reviewed commit: "[CI] Preserve checkout space on single-d..." · Reviewed by Greptile

Comment thread .github/workflows/attach-wheels-to-release.yml Outdated
@ptrendx
ptrendx requested a review from fheinecke August 4, 2026 16:16
@bvolpato
bvolpato force-pushed the bvolpato/publish-gb200-aarch64-wheels branch from cbfc834 to 2da90bf Compare August 16, 2026 02:47
@bvolpato bvolpato changed the title [CI] Publish GB200 aarch64 wheel artifacts [CI] Publish PyTorch 2.11 CUDA 13 release wheels Aug 16, 2026
@bvolpato

Copy link
Copy Markdown
Author

@fheinecke, would you mind taking a look at this when you get a chance? The PR is green and mergeable, and the previous review concern is resolved. It adds PyTorch 2.11/CUDA 13 release wheels for x86_64 and aarch64, with wheel/tag validation and a driverless import check. If someone else owns this release path now, happy to redirect.

@bvolpato

Copy link
Copy Markdown
Author

@ksivaman, could you take a look at this, or point me to current owner for PyTorch release-wheel workflow? It extends #2138 to PyTorch 2.11/CUDA 13 on x86_64 and aarch64. Checks are green and only review thread is resolved.

@bvolpato

Copy link
Copy Markdown
Author

I rechecked this against v2.20.2: the PyTorch 2.11/CUDA 13 aarch64 asset is still missing. The ARM release job fails with fatal: write error: No space left on device during checkout because disk pooling leaves only 32 MiB on the shared root filesystem. I pushed 592be137 to skip pooling when /mnt shares that filesystem. Local layout and build/matrix validation pass, and the PR checks are green. Could someone review the updated release-wheel changes? Native aarch64 compilation and import still need validation on an ARM runner.

@bvolpato

Copy link
Copy Markdown
Author

@fheinecke, would you mind taking a look at this when you get a chance? It adds PyTorch 2.11/CUDA 13 release wheels for x86_64 and aarch64 and fixes the ARM checkout space issue. Checks are green, with native ARM build/import validation still outstanding. If someone else owns the release workflow, happy to redirect.

@0z5a

0z5a commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Some data from the release runs that may help review. Since v2.15, which was the last release with an aarch64 PyTorch wheel attached, the ubuntu-22.04-arm job has failed on every release, and not always for the same reason:

release arm job failure
v2.16, v2.16.1 logs no longer available (#3309 quotes No space left on device for v2.16.1)
v2.17 checkout: No space left on device while cloning 3rdparty/cutlass
v2.17.1 build: RuntimeError: Error compiling objects for extension; the compiler error itself is not in the log
v2.18 build: same as v2.17.1
v2.19 checkout: same as v2.17
v2.20.2 checkout: same as v2.17

So the checkout fix here covers v2.17, v2.19 and v2.20.2. The two compile failures in v2.17.1 and v2.18 cannot be diagnosed from the logs, because the build output went through | tail -n 1. With this PR's logging change, a native ARM run should show whether that failure still happens after the disk fix. It might be worth checking before the next release.

🤖 Generated with Claude Code

@bvolpato

Copy link
Copy Markdown
Author

Thanks for checking the release history. I ran this branch's build scripts on native ARM against v2.20.2, and the existing CUDA 12 and new CUDA 13 wheels built and imported successfully (run). The first CUDA 12 attempt hit a temporary PPA 503, then passed unchanged on retry. I still can't identify the older compiler errors from the truncated logs.

Add Python 3.12 wheels for x86_64 and aarch64 while keeping the
PyTorch 2.8/CUDA 12.9 release builds.

Validate wheel tags and import the new builds before upload.
Keep compiler output visible and propagate build failures.

Signed-off-by: bvolpato <brunocvcunha@gmail.com>
Skip disk pooling and Docker relocation when /mnt shares the root
filesystem. Pooling both paths on ARM runners consumes the space
needed for recursive checkout.

Signed-off-by: bvolpato <brunocvcunha@gmail.com>
@bvolpato
bvolpato force-pushed the bvolpato/publish-gb200-aarch64-wheels branch from 592be13 to 6ab002a Compare October 18, 2026 01:29

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-contribution PRs from external contributor outside the core maintainers, representing community-driven work.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Build] Publish PyTorch 2.11 CUDA 13 aarch64 wheels for GB200

3 participants