Skip to content

Commit

Permalink
fix gen kwargs
Browse files Browse the repository at this point in the history
  • Loading branch information
hiyouga committed May 19, 2024
1 parent 427e7af commit 6844066
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/imitater/model/chat_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ async def _generate(self, messages: List[Dict[str, str]], request_id: str, **gen
sampling_params = SamplingParams(
temperature=gen_kwargs.pop("temperature", self._generation_config.temperature),
top_p=gen_kwargs.pop("top_p", self._generation_config.top_p),
max_tokens=gen_kwargs.pop("max_tokens", self._generation_config.max_new_tokens),
max_tokens=gen_kwargs.pop("max_tokens") or self._generation_config.max_new_tokens,
stop=gen_kwargs.pop("stop", None),
stop_token_ids=self._generation_config.eos_token_id + gen_kwargs.pop("stop_token_ids", []),
)
Expand Down

0 comments on commit 6844066

Please sign in to comment.