feat(inference): add vLLM inference#19

Open

maxreciprocate wants to merge 12 commits intomainfrom

feat-add-vllm-inference

Commits on Aug 30, 2023

Commits on Sep 7, 2023

fix(inference): proper unloading of vllm servers
maxreciprocate
committed

Commits on Sep 12, 2023

feat: add multi-node vllm inference
maxreciprocate
committed

Commits on Sep 13, 2023

feat(inference): add vllm sbatch script
maxreciprocate
committed

Commits on Sep 14, 2023

refactor(inference): update docs, keep hooks under the same api
maxreciprocate
committed

Commits on Sep 15, 2023

Commits on Sep 21, 2023

chore(inference): keep vllm logs in a separate folder
maxreciprocate
committed

Commits on Sep 28, 2023

feat(inference): add RewardHook
maxreciprocate
committed

Commits on Oct 5, 2023

feat(inference): add openai hook
maxreciprocate
committed

Commits on Oct 13, 2023

fix(vllm.sbatch): do `ray stop` & lower `batch_size`
maxreciprocate
committed