[Feature]: Refactor of Sampling Params Override Mechanism #1302
Description
🚀 The feature, motivation and pitch
In #1286 , we see a filter of sampling params keywords which used as a whitelist for overriding default ones. But I think we should use vLLM's sampling params construction mechanism to do this other than additional filter.
Alternatives
No response
Additional context
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.