Skip to content

build(sglang): bump sglang to 0.5.13.post1#3344

Draft
Kh4L wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
Kh4L:bump/sglang-0.5.13.post1
Draft

build(sglang): bump sglang to 0.5.13.post1#3344
Kh4L wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
Kh4L:bump/sglang-0.5.13.post1

Conversation

@Kh4L

@Kh4L Kh4L commented Jul 25, 2026

Copy link
Copy Markdown

Summary

Moves the sglang extra from 0.5.12.post1 to 0.5.13.post1, together with
the pins that have to travel with it.

The motivation is that 0.5.13 is the first release where the chat endpoint
accepts input_ids and returns meta_info.output_token_logprobs, i.e. native
token-in/token-out. Consumers that currently recover tokens by re-tokenizing a
rendered prompt can stop doing that. Everything from 0.5.14 onward adds no
further TITO surface, so this is the cheapest version that delivers it.

sglang               0.5.12.post1  -> 0.5.13.post1
sglang-kernel        0.4.2.post2   -> 0.4.3
sglang-router        (unpinned)    -> ==0.3.2
kernels              >=0.12.0,<0.13 -> >=0.14.1,<0.15
flashinfer-python    0.6.11.post1  -> [cu13]==0.6.12
flashinfer-cubin     0.6.11.post1  -> 0.6.12
flashinfer-jit-cache 0.6.11.post1  -> 0.6.12
transformers         5.6.0         -> 5.8.1

sglang-router was previously unconstrained and already resolving to 0.3.2;
pinning it changes nothing today and closes the floating risk on
launch_router / RouterArgs.

transformers is the interesting one. The sglang extra pinned 5.6.0 while
the mcore extra requires >=5.8.1,<5.9.0, so the two extras disagreed. Both
now agree on 5.8.1, which sits inside the repo-wide <5.9.0 cap. This bump
removes an existing divergence rather than introducing one.

Why not 0.5.15

0.5.15 delivers the same TITO surface and adds two blockers:

  • ServerArgs drops disable_piecewise_cuda_graph, cuda_graph_max_bs and
    cuda_graph_bs at 0.5.14, all three of which this repo passes. They are
    replaced by a different cuda-graph configuration shape, and the existing
    comment in config.py notes that enabling piecewise prefill graphs triggers
    an illegal memory access on this stack — so the rename is not mechanical.
  • 0.5.15 pins transformers==5.12.1, which is unsatisfiable against the base
    transformers>=5.5.0,<5.9.0 in pyproject.toml. Raising that ceiling
    reopens the Megatron-Bridge alignment question and is unrelated to anything
    gained here.

0.5.13.post1 keeps all three cuda-graph fields and pins transformers==5.8.1.

Resolution impact

uv lock moves 12 packages, all inside the sglang closure:

flashinfer-cubin        0.6.11.post1,0.6.8.post1 -> 0.6.12,0.6.8.post1
flashinfer-jit-cache    0.6.11.post1+cu130,0.6.8.post1+cu130 -> 0.6.12+cu130,0.6.8.post1+cu130
flashinfer-python       0.6.11.post1,0.6.12,0.6.8.post1 -> 0.6.12,0.6.8.post1
kernels                 0.12.3 -> 0.14.1
kernels-data            (new)  -> 0.16.0
nvidia-cutlass-dsl      4.5.0,4.5.1,4.5.2 -> 4.5.0,4.5.2
nvidia-cutlass-dsl-libs-cu13  4.5.0,4.5.1,4.5.2 -> 4.5.0,4.5.2
sgl-deep-gemm           0.1.0 -> 0.1.2
sglang                  0.5.12.post1 -> 0.5.13.post1
sglang-kernel           0.4.2.post2 -> 0.4.3
tomlkit                 (new)  -> 0.15.1
transformers            5.5.0,5.5.4,5.6.0,5.8.1 -> 5.5.0,5.5.4,5.8.1

Verified unchanged: torch 2.11.0+cu130, vllm 0.20.0, megatron-bridge,
transformer-engine, nvidia-modelopt, ray 2.56.1, flash-attn, triton,
numpy. The vllm extra keeps flashinfer 0.6.8.post1 and nvidia-cutlass-dsl
4.5.2; only the sglang-side 4.5.1 resolution drops out. uv lock --check is
idempotent afterwards.

aarch64 and x86_64 wheels confirmed for sglang 0.5.13.post1 and sglang-kernel
0.4.3; flashinfer-cubin is py3-none-any.

Validation

ServerArgs compatibility. ServerArgs has 381 annotated fields at
0.5.12.post1 and 396 at 0.5.13.post1. sglang_worker.py::_compute_server_args
passes 36 keys; all 36 are accepted at both versions. The six fields dropped
between the two releases (enable_nan_detection, record_nolora_graph,
nsa_decode_backend, nsa_prefill_backend, nsa_prefill_cp_mode,
enable_nsa_prefill_context_parallel) are none that this repo passes.

Compat patches. _patch_sglang_safe_unpickler and
_patch_sglang_custom_all_reduce_v2_tms_cudagraph were run against real
0.5.13.post1 sources. Both apply cleanly, are idempotent on re-run, and produce
the expected content. The custom-all-reduce TMS backport is still required at
0.5.13 (upstream lands it at 0.5.14), so it is deliberately retained.

GPU, 2 nodes x 4 GPUs (GB200, aarch64), non-colocated Megatron-to-SGLang
broadcast refit
, public Qwen2.5-Math-1.5B-Instruct at a pinned revision:

{"markers": {"world_size": 5, "engines": 4,
             "group_ready_count": 1, "refit_success_count": 3},
 "metrics": {"expected_max_step": 3, "max_recorded_step": 3},
 "status": "passed"}

Three refits, no failure marker, train/loss through step 3. Refit wall time
5.76 s for the first transfer (which builds the communicator), then 0.25 s /
0.25 s — against 5.83 / 0.25 / 0.24 on 0.5.12.post1. No regression.

The run rebuilt its worker environments from this branch's lockfile rather than
reusing the container's prebuilt ones; sglang, sglang-kernel, flashinfer-*
and sgl-deep-gemm were all fetched fresh, and kernels-data — a package that
only appears once these pins are applied — was installed, which is what confirms
the new versions were actually exercised.

To be precise about provenance: that run predates the rebase onto #3339. The
rebase regenerated uv.lock, but the sglang-closure resolution is unchanged by
it — the same 12 packages move to the same versions on either base — so the
measurement still describes the pins in this PR.

Tokenizer behaviour across the transformers move. Because 5.6.0 -> 5.8.1
affects chat-template rendering and tokenization, both were diffed directly:
apply_chat_template(..., tokenize=True), a bare add_special_tokens=False
encode, and a two-render splice-fragment difference, over six conversation
shapes (including <think> content, unicode/CRLF, and a tool-call/tool-response
round trip) each with and without a tool schema, on Qwen3-30B-A3B-Thinking-2507
and Qwen2.5-Math-1.5B-Instruct. 36 cases, zero differences: identical rendered
text, identical token ids, identical vocab size.

Companion change

The SGLang Responses adapter in NVIDIA-NeMo/Gym#1787 pins transformers to
match this extra, and has been moved to 5.8.1 in that PR. This PR does not
touch the Gym submodule and does not depend on that change: the Gym commit
pinned by main does not contain that adapter.

The ordering only matters once both are in: if the Gym change merges while this
one is still open, a Gym checkout would carry a 5.8.1 adapter against a
0.5.12.post1 / 5.6.0 server. The tokenizer diff above shows that particular
combination is behaviourally identical for the models tested, so this is a
consistency note rather than a blocker.

Not covered

  • No x86 GPU run; the hardware used here is aarch64.
  • No image build is included in this PR.
  • The asynchronous NeMo-Gym path is not exercised by the two-node recipe, so the
    transformers move is validated at the tokenizer level (above) rather than
    through a full Gym rollout.

@copy-pr-bot

copy-pr-bot Bot commented Jul 25, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Moves the sglang extra to 0.5.13.post1 and the pins that must travel with it:
sglang-kernel 0.4.3, kernels >=0.14.1,<0.15, flashinfer 0.6.12 (with the new
cu13 extra on flashinfer-python), and transformers 5.8.1. Also pins
sglang-router, which was floating and already resolving to 0.3.2.

transformers 5.8.1 is the notable one: the sglang extra was pinned to 5.6.0
while the mcore extra requires >=5.8.1,<5.9.0, so the two extras disagreed.
Both now agree on 5.8.1, which sits inside the repo-wide <5.9.0 cap.

Resolution moves 12 packages, all within the sglang closure. torch, vllm,
megatron-bridge, transformer-engine, nvidia-modelopt, ray, flash-attn, triton
and numpy are unchanged, and the vllm extra keeps flashinfer 0.6.8.post1 and
nvidia-cutlass-dsl 4.5.2.

Signed-off-by: Serge Panev <spanev@nvidia.com>
@Kh4L
Kh4L force-pushed the bump/sglang-0.5.13.post1 branch from 71140b4 to ed30fcc Compare July 25, 2026 03:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant