The inference results appear to be a repetition of several words

[SZlulingyi]

i am using the model“pentagoniac/SEMIKONG-8b-GPTQ” and using "python -m vllm.entrypoints.api_server --model /public/home/lulingyi/repo/semikong/model/SEMIKONG-8b-GPTQ --device cuda --max-lora-rank 32 --dtype float16 --port 8080".
When I’m making inferences, the responses seem to be repetitive, consisting of the same few words or numbers. How can I address this issue?

[mfq2003]

hello, where did you get the file:pentagoniac/SEMIKONG-8b-GPTQ?
I even did not find it!
can you help me, please?