Skip to content

Deprecated vLLM parameters in V1 engine #697

@giladfrid009

Description

@giladfrid009

Currently in your EngineArgs class there are several vllm engine arguments that have been deprecated in V1 engine, and therefore cause crashes since they don't exist anymore.

class EngineArgs(TypedDict, total=False):
    model: str
    served_model_name: str | list[str] | None
    tokenizer: str | None
    # task: TaskOption
    skip_tokenizer_init: bool
    tokenizer_mode: str
...

For example:

The result of running your temporal clue example with the following config:

model = art.TrainableModel(
    name="066", project="temporal-clue", base_model="Qwen/Qwen2.5-7B-Instruct",
    _internal_config=InternalModelConfig(engine_args=EngineArgs(num_scheduler_steps=1)),
)

The num_scheduler_steps parameter does not exist anymore.

File ~/source/AgentDaC/.venv/lib/python3.11/site-packages/art/unsloth/service.py:1020, in llm()
   1018 for key in ["enable_log_requests", "disable_log_requests"]:
   1019     engine_args.pop(key, None)
-> [1020](https://vscode-remote+tunnel-002bgalileo1.vscode-resource.vscode-cdn.net/home/fre.gilad/source/AgentDaC/notebooks/examples/temporal_clue/~/source/AgentDaC/.venv/lib/python3.11/site-packages/art/unsloth/service.py:1020) return asyncio.create_task(get_llm(AsyncEngineArgs(**engine_args)))

TypeError: AsyncEngineArgs.__init__() got an unexpected keyword argument 'num_scheduler_steps'

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions