vllm-edge

The edge-based vLLM framework, which is suitable for devices with limited video memory, low communication bandwidth, and weak computing ability.

install

git clone https://github.com/JingliangGao/vllm-edge.git
cd vllm-edge/
chmod +x ./build-for-debug.sh && ./build-for-debug.sh

modelscope download --model Qwen/Qwen3-0.6B

cd vllm_edge/examples/
python3  qwen3_inference.py

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
benchmarks		benchmarks
examples		examples
vllm_edge		vllm_edge
LICENSE		LICENSE
README.md		README.md
build-for-debug.sh		build-for-debug.sh
requirements.txt		requirements.txt
setup.py		setup.py