Skip to content

feat(config): expose vLLM enforce_eager and compilation_config #698

Description

@binaryaaron

Is your feature request related to a problem?

Safe Synthesizer exposes generation.attention_backend, but operators cannot configure vLLM's enforce_eager or compilation_config without patching VllmBackend.

The current parameter model ends with the attention-backend control (GenerateParameters). Engine initialization passes that setting to vLLM but leaves eager execution and compilation at vLLM's defaults (VllmBackend.initialize).

This limits runtime debugging and compatibility work. vLLM 0.24 accepts both controls on LLM: enforce_eager and compilation_config.

PR #363 removed an internally derived enforce_eager value after subprocess isolation resolved the Unsloth conflict. The requested configuration should restore operator control without coupling eager mode to a training implementation.

Describe the solution you'd like

Add typed generation settings that preserve current behavior by default:

generation:
  enforce_eager: false
  compilation_config: null

Expected types:

  • generation.enforce_eager: bool = false
  • generation.compilation_config: int | dict[str, Any] | None = null

Pass both values through to the vLLM LLM constructor. A mapping should remain an atomic configuration leaf so vLLM owns validation of its version-specific compilation fields.

Document an example that disables model-graph compilation:

generation:
  enforce_eager: true
  compilation_config:
    mode: 0

Define the CLI behavior for the mapping field explicitly. YAML and SDK inputs must support the full mapping form.

Acceptance criteria

  • GenerateParameters exposes both fields with defaults matching current vLLM behavior.
  • VllmBackend.initialize() forwards resolved values to vLLM.
  • Existing configurations produce the same engine arguments as before.
  • Unit tests cover defaults and explicit overrides, including a mapping-valued compilation_config.
  • Configuration reference and running guidance explain eager-mode performance cost and the accepted compilation-config forms.
  • Resume-time generation overrides accept both settings through the existing generation section.

Describe alternatives you've considered

Users can maintain a source patch or construct vLLM outside Safe Synthesizer. Both approaches bypass the supported YAML, CLI, and SDK configuration flow and make saved-run generation harder to reproduce.

Additional context

This control does not remove every runtime compilation requirement. Triton compiles its CUDA host-driver shim on first GPU use, so enforce_eager: true does not replace the compiler dependency fixed by #682. Documentation should keep that boundary clear.

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions