-
Notifications
You must be signed in to change notification settings - Fork 59
Add workflow to build vLLM-TPU wheel using PyPI tpu-inference #1241
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
3c1e18f to
87a31c2
Compare
Signed-off-by: Ylang Tsou <[email protected]>
edc52f7 to
96aabe7
Compare
Signed-off-by: Ylang Tsou <[email protected]>
5967ddd to
981ba3b
Compare
|
I think it's pretty good. Just a few things I want to mention. It seems like you haven't included the And, please provide two types of tests: Please update the PR description as well, as some information might be outdated. And we don't need to mention every file's changes, a brief summary of the PR is sufficient. |
Description
A new workflow to build and test vllm-tpu using the nightly tpu-inference package from PyPI, instead of building it from source and verify that the wheel builds and installs successfully, with vllm serve starting up correctly in an E2E environment.
Key Changes:
(New) pipeline_pypi.yml:
Added a 20-minute delay step to allow time for the tpu-inference package to be published, as the schedules currently overlap, before running the benchmark tests.
(New) build_vllm_tpu.sh:
Builds the vLLM-TPU wheel by automating the cloning and patching process.
Usage: ./build_vllm_tpu.sh [vllm-branch-or-tag]
(New) Dockerfile.pypi:
Builds the vllm-tpu wheel using the nightly tpu-inference version inside the container and installs it for benchmark testing.
(Updated) setup_docker_env.sh
Added logic to switch DOCKERFILE_NAME to Dockerfile.pypi when the RUN_WITH_PYPI environment variable is set to true.
(New) run_with_pypi.sh
New entry point script that sets RUN_WITH_PYPI="true" and calls run_in_docker.sh
Tests
Test on buildkite
Checklist
Before submitting this PR, please make sure: