[Klaud Cold] Update qwen3.5-fp8-b300-sglang (+mtp) SGLang image to v0.5.12-cu130#1451
[Klaud Cold] Update qwen3.5-fp8-b300-sglang (+mtp) SGLang image to v0.5.12-cu130#1451functionstackx wants to merge 4 commits into
Conversation
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
1 similar comment
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26006160501 |
There was a problem hiding this comment.
LGTM, straightforward SGLang image version bump with matching changelog entry.
Extended reasoning...
Overview
This PR bumps the SGLang image for two B300 configurations (qwen3.5-fp8-b300-sglang and qwen3.5-fp8-b300-sglang-mtp) to v0.5.12-cu130 in .github/configs/nvidia-master.yaml, plus a corresponding entry in perf-changelog.yaml. Total diff is 4 lines of config and 7 lines of changelog.
Security risks
None. This is a configuration change to container image tags used by the perf sweep, not production-serving code, and no auth/crypto/permissions surface is touched.
Level of scrutiny
Low. Image version bumps with a matching changelog entry are mechanical and follow the established pattern in recent commits (e.g., #1404, #1394, #1416). The full-sweep-enabled label means the full sweep will validate the new image works for these recipes.
Other factors
The bug hunting system found no issues, there are no outstanding human reviewer comments, and the two automated recipe-reminder bot comments are boilerplate that don't require action. No prior reviews from me on this PR.
Same workaround as PR #1422 — bypass the broken flash-attn cute kernel sm_103 assertion in the Qwen-3.5-VL vision encoder by switching only the multi-modal attention path to triton_attn. Text decoder still uses --attention-backend trtllm_mha. See sgl-project/sglang#25564 + Dao-AILab/flash-attention#2572 for the upstream root cause and the in-flight fix.
Trying the same workaround as PR #1422:
|
# Conflicts: # perf-changelog.yaml
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26006162286 |
Summary
qwen3.5-fp8-b300-sglangfromlmsysorg/sglang:v0.5.10.post1-cu130(30d old) tolmsysorg/sglang:v0.5.12-cu130.qwen3.5-fp8-b300-sglang-mtpfromlmsysorg/sglang:v0.5.11-cu130tolmsysorg/sglang:v0.5.12-cu130.Test plan
full-sweep-enabledlabel.🤖 Generated with Claude Code