feat(generation): add TensorRT-LLM rollout backend#2420

Open

shuyixiong wants to merge 56 commits into

NVIDIA-NeMo:mainfrom

shuyixiong:shuyix/trtllm

Commits on Jul 23, 2026

feat(trtllm): add TRT-LLM generation backend

authored and
joyang-nv
committed
Custom build wheel

shuyixiong
authored and
joyang-nv
committed
build(trtllm): migrate from UV_FIND_LINKS stub wheel to PEP 517 path source

shuyixiong
authored and
joyang-nv
committed
fix(trtllm): correct workspace deps and scope cutlass override to vllm

shuyixiong
authored and
joyang-nv
committed
Pin nvidia-cutlass-dsl version in vllm

shuyixiong
authored and
joyang-nv
committed
Drop custom modelopt build

shuyixiong
authored and
joyang-nv
committed
Fix eos_id issue

shuyixiong
authored and
joyang-nv
committed
Rename trtllm workspace

shuyixiong
authored and
joyang-nv
committed
Clean code

shuyixiong
authored and
joyang-nv
committed
allow TRT-LLM backend for async GRPO

shikicloud
authored and
joyang-nv
committed
Deprecate sync trtllm_worker code

shuyixiong
authored and
joyang-nv
committed
refactor(trtllm): remove TRT-LLM HTTP server (to be added back in the agentRL PR)

shuyixiong
authored and
joyang-nv
committed
Clean code

shuyixiong
authored and
joyang-nv
committed
Resolve rebase conflict

shuyixiong
authored and
joyang-nv
committed
Reorder uv sync step and move cuDNN env setup before TE build

shuyixiong
authored and
joyang-nv
committed
Resolve review comment

shuyixiong
authored and
joyang-nv
committed
Fix copyright-check and pre-commit changes

shuyixiong
authored and
joyang-nv
committed
fixup! Resolve review comment

shuyixiong
authored and
joyang-nv
committed
Add dtensor nightly tests and release test

shuyixiong
authored and
joyang-nv
committed
Add trtllm setup dependency

shuyixiong
authored and
joyang-nv
committed
add ccache

shikicloud
authored and
joyang-nv
committed
fix(recipe): disable async checkpoint save for nanov3-30BA3B trtllm to avoid colocated wake OOM

shuyixiong
authored and
joyang-nv
committed
add trtllm ci test

shikicloud
authored and
joyang-nv
committed
Resolve rebase conflict

shuyixiong
authored and
joyang-nv
committed
Change description on CI test

shikicloud
authored and
joyang-nv
committed
Pre-commit changes

shuyixiong
authored and
joyang-nv
committed
Apply the taplo-format hook's changes manually

shuyixiong
authored and
joyang-nv
committed
Add newly added files to pyrefly.toml

shuyixiong
authored and
joyang-nv
committed
Add IPC-ZMQ refit unit tests

shikicloud
authored and
joyang-nv
committed
Resolve review comment

shuyixiong
authored and
joyang-nv
committed
Resolve review comment

shuyixiong
authored and
joyang-nv
committed
Minor fix

shuyixiong
authored and
joyang-nv
committed
Fix rebase conflict and Add logprobs_simple_format=True

shuyixiong
authored and
joyang-nv
committed
build(trtllm): single-source the fork ref in [tool.trtllm]

shuyixiong
authored and
joyang-nv
committed
Reduce compile jobs to reduce memory pressure

shuyixiong
authored and
joyang-nv
committed
validate ccache reuse

shikicloud
authored and
joyang-nv
committed
Resolve rebase conflict

shuyixiong
authored and
joyang-nv
committed
Add TRT-LLM non-colocated async recipe

shikicloud
authored and
joyang-nv
committed
build: refresh uv lock

shikicloud
authored and
joyang-nv
committed
Support save partial ccache and fix L0 bugs

shikicloud
authored and
joyang-nv
committed
ci: reuse TRT-LLM ccache by base image
joyang-nv
committed
ci: cache TRT-LLM uv builds
joyang-nv
committed
ci: reduce TRT-LLM build log noise
joyang-nv
committed
ci: focus TRT-LLM build logging
joyang-nv
committed
ci: finalize TRT-LLM build caching
joyang-nv
committed
ci: skip TRT-LLM cache work when disabled
joyang-nv
committed
ci: optimize TRT-LLM cache reuse
joyang-nv
committed
ci: simplify TRT-LLM build caching
joyang-nv
committed
ci: document TRT-LLM lockfile correctness guard
joyang-nv
committed
ci: clarify late lockfile consistency check
joyang-nv
committed
ci: skip empty TRT-LLM cache deltas
joyang-nv
committed
ci: reuse prebuilt TRT-LLM wheel in L0 tests
joyang-nv
committed
test: pass required NeMo Gym rollout arguments
joyang-nv
committed
Revert "test: pass required NeMo Gym rollout arguments"
joyang-nv
committed
test: consume async NeMo Gym rollout results

RayenTian
authored and
joyang-nv
committed
Merge branch 'main' into shuyix/trtllm
joyang-nv
authored