[Installation]: getting vLLM installed with a free-threaded Python interpreter (3.14t)

[rgommers]

https://x.com/vllm_project/status/1942450223881605593 showed that it's possible to run vLLM in a free-threaded Python interpreter. That involved a lot of custom work to get dependencies to build, and the situation now (4 months later) is a lot better. This is meant as a tracking issue for all the dependencies of vLLM - to start with on Linux x86-64 (CPU and CUDA) - and as a "work list" for getting the remaining issues with those dependencies resolved. With a goal of uv pip install vllm to do the right thing out of the box in a clean 3.14t environment.

Python 3.14t is necessary, both because CPython 3.14t itself is much more stable than 3.13t and because there are a number of important packages (e.g., cffi, aiohttp) that do support 3.14t but won't support 3.13t. Of course this depends on vLLM itself supporting Python 3.14 (the default, with-GIL build) first - see gh-26994 gh-34096 for that.

This is the dependency graph for vLLM, generated from the default dependencies for the latest release (0.11.0) on PyPI, with packages with compiled code with free-threading wheels on PyPI marked in green and those without in red (easier to browse in a fresh browser tab):

PyTorch 2.9.0 has 3.14/3.14t wheels marked as "preview"; 2.10.0 in January will contain full support.

On Linux x86-64, here are all the packages from the build, cpu and common requirements files that don't cleanly install from PyPI:

• msgspec: has support in its main branch, installs fine from source. EDIT 27 Nov '25: msgspec 0.20.0 has support and wheels on PyPI.
• numba: a release with support is being worked on, see numba#9928. Release planned for January says the Numba team.
  • numba 0.65.0 added full cp314t support and wheels
• opencv-python-headless: no support yet, tracking issue is opencv#27933
  • To build from source: clone the repo, check out opencv-python#1051, ensure the build dependencies in pyproject.toml are installed (latest versions, ignore the pins) and then build with export ENABLE_HEADLESS=1 && uv pip install . --no-build-isolation
• outlines-core: latest version (0.2.14) builds fine from source after outlines-core#235, and the next release will have cp314t wheels (xref outlines-core#248)
• llguidance: doesn't build now builds from source on main after llguidance#255, needs some fixes (xref llguidance#256)
  • llguidance >=1.6.0 has complete support and cp314t wheels
• openai-harmony: latest version on PyPI builds fine from source (xref harmony#87 for support/wheels)
• safetensors: builds from source (main branch)
• xgrammar: latest version (0.1.27) builds fine from source on Linux. xref xgrammar#500 for full support/wheels. EDIT: 0.1.31 has full support and cp314t wheels
• tokenizers: latest version on PyPI builds fine from source

And the CUDA-specific dependencies that don't yet have support:

• flashinfer-python: no upstream issue yet; uses stable ABI so doesn't build yet resolved by misc: Do not use the limited API with free-threaded Python flashinfer-ai/flashinfer#1687
• ray: revisit after 3.14 support (xref ray#56434) has landed
  • ray was made optional, so this is no longer blocking. Ray still doesn't even have regular 3.14 support as of 10 April 2026 fixed on 22 April 2026.
• xformers: no support yet, not expected soon (xref xformers#1345).
  • Resolved in xformers 0.0.35 by removing any dependency on the CPython C API.

All other packages install fine. Note that there are dependencies that have optional C extensions that aren't yet working so those packages will be pulled in as pure Python currently (e.g., cbor2 and protobuf). And there are packages with known thread-safety issues (e.g., msgpack, protobuf). Those packages may have wheels or seem to install fine, but do not yet have a check in the "PyPI release" column in https://py-free-threading.github.io/tracking/.

So as of today, everything except for llguidance is installable from PyPI or from source with a modest bit of effort. Something along these lines for cpu:

uv venv --python=3.14t venv-vllm
source venv-vllm/bin/activate
uv pip install torch torchaudio torchvision triton --torch-backend cpu
# Comment out `torch` in `build.txt`
uv pip install -r requirements/build.txt

# Now install the dependencies in the checklist above one by one
# You need GCC and a Rust compiler installed 

# Comment out `llguidance` in `common.txt`; also need to loose remove some == pins
uv pip install -r requirements/common.txt

After all that is done, it should be possible to build vLLM itself. PRs to get that to work:

• vllm#29241
• flash-attention#112

[ngoldbaum]

Now that PyTorch 2.10 is out, I think it should be possible to add some content to the vLLM docs that guides users through how to set up vLLM on the free-threaded build by building dependencies from source.

[nascheme]

Two PRs that help with testing with the free-threaded Python build:

• [Feature] Lazy import for the "mistral" tokenizer module. #34651
• [Feature] Lazy import of ngram_proposer module. #34654

The first is because opencv does not yet support the free-threaded build and Mistral requires that. The second is because numba does not yet support free-threaded (hopefully real soon now) and ngram_proposer needs numba.

I think these two changes are worth doing in any case since they should reduce startup time if you are not using that functionality.

[rgommers]

Status update:

• Full Python 3.14 support (default, with-GIL interpreter) is now complete, [Tracking Feature]: Support vLLM with Python 3.14 #34096 was just closed.
• NVIDIA packages like cuda-tile and nvidia-cutlass-dsl were released with 3.14t support, others from NVIDIA are actively being worked on.
• ray was made optional, so can be ignored for the time being. Given how long it took them to do a 3.14 release, I wouldn't expect 3.14t support any time soon.
• outlines-core, safetensors and tokenizers changed landed already, they just need to do releases
• openai-harmony and opencv-headless-python are the only two where not much is moving; unresponsive maintainers unfortunately.

[reneleonhardt]

Great to see such progress! ❤️

Can free-threading support (this ticket) be expected in the future too, after dependencies hopefully created releases too?

When will Python 3.15 support be tackled (never ending release stream)? 😄
https://blog.python.org/2026/05/python-3150-beta-1/
Already available as setup-python 3.15-dev and 3.15t-dev

[rgommers]

Can free-threading support (this ticket) be expected in the future too, after dependencies hopefully created releases too?

That's the goal, but at the current pace - and also keeping in mind that new dependencies keep being added that might also need work - it's going to be a while (>3 months). It's already not too difficult anymore to get vLLM and its dependencies installed though, just needs building the few that miss wheels from source. https://github.com/nascheme/vllm-ft-build is quite helpful for that. Running vLLM unchanged under a free-threaded interpreter will very likely "just work" for you. The interesting bit is of course starting to use multi-threading, which is also where you might run into sharp edges still.

When will Python 3.15 support be tackled

I'd keep that separate. Given that 3.14 support was just released, I'd expect it to take about a year (or >8 months at a minimum).

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!