vllm can not start normally with multi 4090 gpus

[lddlww]

code:

from vllm import LLM
llm=LLM(model="/data/models/Qwen/Qwen2.5-14B-Instruct-1M",tensor_parallel_size=2)
output=llm.generate("hello")
print(output)

run code:

python3 test.py

but vllm can not output response, and paused by vLLM is using nccl==2.21.5 log

[comaniac]

4090 doesn't enable P2P by default. You can try

export CUDA_P2P_ENABLED=1

But in general it could be very slow because this P2P is via PCIe

[Alejsey]

you need to specify pipeline_parallel_size