build(sglang): bump sglang to 0.5.13.post1#3344
Draft
Kh4L wants to merge 1 commit into
Draft
Conversation
Moves the sglang extra to 0.5.13.post1 and the pins that must travel with it: sglang-kernel 0.4.3, kernels >=0.14.1,<0.15, flashinfer 0.6.12 (with the new cu13 extra on flashinfer-python), and transformers 5.8.1. Also pins sglang-router, which was floating and already resolving to 0.3.2. transformers 5.8.1 is the notable one: the sglang extra was pinned to 5.6.0 while the mcore extra requires >=5.8.1,<5.9.0, so the two extras disagreed. Both now agree on 5.8.1, which sits inside the repo-wide <5.9.0 cap. Resolution moves 12 packages, all within the sglang closure. torch, vllm, megatron-bridge, transformer-engine, nvidia-modelopt, ray, flash-attn, triton and numpy are unchanged, and the vllm extra keeps flashinfer 0.6.8.post1 and nvidia-cutlass-dsl 4.5.2. Signed-off-by: Serge Panev <spanev@nvidia.com>
Kh4L
force-pushed
the
bump/sglang-0.5.13.post1
branch
from
July 25, 2026 03:33
71140b4 to
ed30fcc
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Moves the
sglangextra from 0.5.12.post1 to 0.5.13.post1, together withthe pins that have to travel with it.
The motivation is that 0.5.13 is the first release where the chat endpoint
accepts
input_idsand returnsmeta_info.output_token_logprobs, i.e. nativetoken-in/token-out. Consumers that currently recover tokens by re-tokenizing a
rendered prompt can stop doing that. Everything from 0.5.14 onward adds no
further TITO surface, so this is the cheapest version that delivers it.
sglang-routerwas previously unconstrained and already resolving to 0.3.2;pinning it changes nothing today and closes the floating risk on
launch_router/RouterArgs.transformersis the interesting one. Thesglangextra pinned 5.6.0 whilethe
mcoreextra requires>=5.8.1,<5.9.0, so the two extras disagreed. Bothnow agree on 5.8.1, which sits inside the repo-wide
<5.9.0cap. This bumpremoves an existing divergence rather than introducing one.
Why not 0.5.15
0.5.15 delivers the same TITO surface and adds two blockers:
ServerArgsdropsdisable_piecewise_cuda_graph,cuda_graph_max_bsandcuda_graph_bsat 0.5.14, all three of which this repo passes. They arereplaced by a different cuda-graph configuration shape, and the existing
comment in
config.pynotes that enabling piecewise prefill graphs triggersan illegal memory access on this stack — so the rename is not mechanical.
transformers==5.12.1, which is unsatisfiable against the basetransformers>=5.5.0,<5.9.0inpyproject.toml. Raising that ceilingreopens the Megatron-Bridge alignment question and is unrelated to anything
gained here.
0.5.13.post1 keeps all three cuda-graph fields and pins
transformers==5.8.1.Resolution impact
uv lockmoves 12 packages, all inside the sglang closure:Verified unchanged:
torch2.11.0+cu130,vllm0.20.0,megatron-bridge,transformer-engine,nvidia-modelopt,ray2.56.1,flash-attn,triton,numpy. Thevllmextra keeps flashinfer 0.6.8.post1 and nvidia-cutlass-dsl4.5.2; only the sglang-side 4.5.1 resolution drops out.
uv lock --checkisidempotent afterwards.
aarch64 and x86_64 wheels confirmed for sglang 0.5.13.post1 and sglang-kernel
0.4.3;
flashinfer-cubinispy3-none-any.Validation
ServerArgscompatibility.ServerArgshas 381 annotated fields at0.5.12.post1 and 396 at 0.5.13.post1.
sglang_worker.py::_compute_server_argspasses 36 keys; all 36 are accepted at both versions. The six fields dropped
between the two releases (
enable_nan_detection,record_nolora_graph,nsa_decode_backend,nsa_prefill_backend,nsa_prefill_cp_mode,enable_nsa_prefill_context_parallel) are none that this repo passes.Compat patches.
_patch_sglang_safe_unpicklerand_patch_sglang_custom_all_reduce_v2_tms_cudagraphwere run against real0.5.13.post1 sources. Both apply cleanly, are idempotent on re-run, and produce
the expected content. The custom-all-reduce TMS backport is still required at
0.5.13 (upstream lands it at 0.5.14), so it is deliberately retained.
GPU, 2 nodes x 4 GPUs (GB200, aarch64), non-colocated Megatron-to-SGLang
broadcast refit, public Qwen2.5-Math-1.5B-Instruct at a pinned revision:
{"markers": {"world_size": 5, "engines": 4, "group_ready_count": 1, "refit_success_count": 3}, "metrics": {"expected_max_step": 3, "max_recorded_step": 3}, "status": "passed"}Three refits, no failure marker,
train/lossthrough step 3. Refit wall time5.76 s for the first transfer (which builds the communicator), then 0.25 s /
0.25 s — against 5.83 / 0.25 / 0.24 on 0.5.12.post1. No regression.
The run rebuilt its worker environments from this branch's lockfile rather than
reusing the container's prebuilt ones;
sglang,sglang-kernel,flashinfer-*and
sgl-deep-gemmwere all fetched fresh, andkernels-data— a package thatonly appears once these pins are applied — was installed, which is what confirms
the new versions were actually exercised.
To be precise about provenance: that run predates the rebase onto #3339. The
rebase regenerated
uv.lock, but the sglang-closure resolution is unchanged byit — the same 12 packages move to the same versions on either base — so the
measurement still describes the pins in this PR.
Tokenizer behaviour across the
transformersmove. Because 5.6.0 -> 5.8.1affects chat-template rendering and tokenization, both were diffed directly:
apply_chat_template(..., tokenize=True), a bareadd_special_tokens=Falseencode, and a two-render splice-fragment difference, over six conversation
shapes (including
<think>content, unicode/CRLF, and a tool-call/tool-responseround trip) each with and without a tool schema, on Qwen3-30B-A3B-Thinking-2507
and Qwen2.5-Math-1.5B-Instruct. 36 cases, zero differences: identical rendered
text, identical token ids, identical vocab size.
Companion change
The SGLang Responses adapter in NVIDIA-NeMo/Gym#1787 pins
transformerstomatch this extra, and has been moved to 5.8.1 in that PR. This PR does not
touch the Gym submodule and does not depend on that change: the Gym commit
pinned by
maindoes not contain that adapter.The ordering only matters once both are in: if the Gym change merges while this
one is still open, a Gym checkout would carry a 5.8.1 adapter against a
0.5.12.post1 / 5.6.0 server. The tokenizer diff above shows that particular
combination is behaviourally identical for the models tested, so this is a
consistency note rather than a blocker.
Not covered
transformersmove is validated at the tokenizer level (above) rather thanthrough a full Gym rollout.