Skip to content
This repository was archived by the owner on Mar 5, 2025. It is now read-only.

Compatibiliy with upstream vllm-project/vllm repository #5

Merged
yannicks1 merged 5 commits into from
Feb 21, 2025
Merged

Compatibiliy with upstream vllm-project/vllm repository #5

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
4be4bc8
add get_model function as required by abstract base class
yannicks1 Feb 20, 2025
a77ebd9
formatting yapf
yannicks1 Feb 20, 2025
6c13d9e
bump scheduler to vLLM v.0.7.3
yannicks1 Feb 21, 2025
ac12781
update Docker file to install vllm upstream release v0.7.3
yannicks1 Feb 21, 2025
38f1329
update readme to install vllm version from wheel
yannicks1 Feb 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/test-spyre.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,16 @@ jobs:
run: |
docker run -i --rm --entrypoint /bin/bash vllm-spyre -c '''
pip install pytest sentence-transformers && \
python3.12 -c "from transformers import pipeline; pipeline(\"text-generation\", model=\"JackFram/llama-160m\")" && \
python -c "from transformers import pipeline; pipeline(\"text-generation\", model=\"JackFram/llama-160m\")" && \
export VARIANT=$(ls /root/.cache/huggingface/hub/models--JackFram--llama-160m/snapshots/) && \
mkdir -p /models && \
ln -s /root/.cache/huggingface/hub/models--JackFram--llama-160m/snapshots/${VARIANT} /models/llama-194m && \
python3.12 -c "from sentence_transformers import SentenceTransformer; SentenceTransformer(\"sentence-transformers/all-roberta-large-v1\")" && \
python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer(\"sentence-transformers/all-roberta-large-v1\")" && \
export VARIANT=$(ls /root/.cache/huggingface/hub/models--sentence-transformers--all-roberta-large-v1/snapshots/) && \
ln -s /root/.cache/huggingface/hub/models--sentence-transformers--all-roberta-large-v1/snapshots/${VARIANT} /models/all-roberta-large-v1 && \
export MASTER_PORT=12355 && \
export MASTER_ADDR=localhost && \
export DISTRIBUTED_STRATEGY_IGNORE_MODULES=WordEmbedding && \
cd vllm-spyre && \
python3.12 -m pytest tests -v
python -m pytest tests -v
'''
11 changes: 4 additions & 7 deletions Dockerfile.spyre
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,17 +12,14 @@ WORKDIR /workspace
RUN microdnf update -y && microdnf install -y \
python${PYTHON_VERSION}-devel python${PYTHON_VERSION}-pip python${PYTHON_VERSION}-wheel git vim gcc g++ kmod\
&& microdnf clean all
RUN ln -sf $(which python${PYTHON_VERSION}) /usr/bin/python && \
ln -sf $(which pip${PYTHON_VERSION}) /usr/bin/pip

# Download and install vllm ###########################################################
ENV VLLM_TARGET_DEVICE=empty
RUN git clone -b sop-plugin-refactoring --single-branch https://github.com/IBM/vllm.git \
&& cd vllm \
&& python3.12 -m pip install --upgrade pip \
&& pip install -r requirements-build.txt \
&& pip install --no-build-isolation -v -e . \
&& mkdir /workspace/vllm-spyre
RUN pip install vllm==0.7.3

# Install vllm Spyre plugin ##################################################################
RUN mkdir /workspace/vllm-spyre
COPY . /workspace/vllm-spyre
RUN cd /workspace/vllm-spyre && pip install --no-build-isolation -v -e .
ENV VLLM_PLUGINS=spyre
Expand Down
Loading