Skip to content

[Klaud Cold] Update qwen3.5-fp8-b300-sglang (+mtp) SGLang image to v0.5.12-cu130#1451

Open
functionstackx wants to merge 4 commits into
mainfrom
update-qwen3.5-fp8-b300-sglang-v0.5.12
Open

[Klaud Cold] Update qwen3.5-fp8-b300-sglang (+mtp) SGLang image to v0.5.12-cu130#1451
functionstackx wants to merge 4 commits into
mainfrom
update-qwen3.5-fp8-b300-sglang-v0.5.12

Conversation

@functionstackx
Copy link
Copy Markdown
Collaborator

Summary

  • Bumps qwen3.5-fp8-b300-sglang from lmsysorg/sglang:v0.5.10.post1-cu130 (30d old) to lmsysorg/sglang:v0.5.12-cu130.
  • Bumps qwen3.5-fp8-b300-sglang-mtp from lmsysorg/sglang:v0.5.11-cu130 to lmsysorg/sglang:v0.5.12-cu130.

Test plan

  • Full sweep passes with full-sweep-enabled label.

🤖 Generated with Claude Code

@github-actions
Copy link
Copy Markdown
Contributor

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

1 similar comment
@github-actions
Copy link
Copy Markdown
Contributor

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow

As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers.

If additional help is needed, PR authors can reach out to core maintainers over Slack.

@github-actions
Copy link
Copy Markdown
Contributor

Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, straightforward SGLang image version bump with matching changelog entry.

Extended reasoning...

Overview

This PR bumps the SGLang image for two B300 configurations (qwen3.5-fp8-b300-sglang and qwen3.5-fp8-b300-sglang-mtp) to v0.5.12-cu130 in .github/configs/nvidia-master.yaml, plus a corresponding entry in perf-changelog.yaml. Total diff is 4 lines of config and 7 lines of changelog.

Security risks

None. This is a configuration change to container image tags used by the perf sweep, not production-serving code, and no auth/crypto/permissions surface is touched.

Level of scrutiny

Low. Image version bumps with a matching changelog entry are mechanical and follow the established pattern in recent commits (e.g., #1404, #1394, #1416). The full-sweep-enabled label means the full sweep will validate the new image works for these recipes.

Other factors

The bug hunting system found no issues, there are no outstanding human reviewer comments, and the two automated recipe-reminder bot comments are boilerplate that don't require action. No prior reviews from me on this PR.

@functionstackx functionstackx changed the title Update qwen3.5-fp8-b300-sglang (+mtp) SGLang image to v0.5.12-cu130 [Klaud Cold] Update qwen3.5-fp8-b300-sglang (+mtp) SGLang image to v0.5.12-cu130 May 17, 2026
Same workaround as PR #1422 — bypass the broken flash-attn cute kernel
sm_103 assertion in the Qwen-3.5-VL vision encoder by switching only
the multi-modal attention path to triton_attn. Text decoder still uses
--attention-backend trtllm_mha.

See sgl-project/sglang#25564 + Dao-AILab/flash-attention#2572 for the
upstream root cause and the in-flight fix.
@functionstackx
Copy link
Copy Markdown
Collaborator Author

Trying the same workaround as PR #1422: --mm-attention-backend triton_attn

This PR hits the same flash-attn-4 cute kernel sm_103 assertion as the sibling bf16 b300 PR. Per the PR #1422 comment thread, adding --mm-attention-backend triton_attn bypasses the broken flash-attn path for the multi-modal (vision encoder) layer while keeping --attention-backend trtllm_mha for the text decoder.

Pushed as 3e3313bc (touches both qwen3.5_fp8_b300.sh and qwen3.5_fp8_b300_mtp.sh).

Upstream / root cause cross-links

  • sgl-project/sglang#25564 — root cause is the cutedsl Arch enum aliasing differently between cu13 and non-cu13 paths, so Arch.sm_110f ends up < sm_103 and the assertion rejects B300 / B300-ultra
  • @mmangkad's comment — proposed durable fix is bumping sglang's nvidia-cutlass-dsl dependency to the [cu13] extra; the FA PR isn't strictly needed
  • Dao-AILab/flash-attention#2572is_family_of()-based assertion fix (awaiting @tridao review)

Once either upstream lands, this PR can drop the workaround. In the meantime the triton_attn swap keeps the sweep alive. Fallback if that still fails: pin back to v0.5.11-cu130.

# Conflicts:
#	perf-changelog.yaml
@github-actions
Copy link
Copy Markdown
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant