Skip to content

bug: ModelBooster with vLLM backend gets stuck in Failed when workers[].pods is omitted (generates workerReplicas: -1) #1612

Description

@vanshika2720

Description

When using a vLLM ModelBooster, omitting workers[].pods can cause the generated ModelServing to have an invalid workerReplicas: -1.

The pods field is optional, so a user can reasonably leave it unset. However, the ModelBooster currently treats the omitted value as 0 and calculates the number of worker replicas as pods - 1. This results in -1, which is rejected by the ModelServing validation webhook.

The ModelBooster then enters Failed status and keeps retrying the same invalid configuration. It does not recover unless the user discovers and fixes the underlying pods configuration.

Steps to Reproduce

Create a ModelBooster using the vLLM backend and omit pods from the worker configuration:

apiVersion: workload.serving.volcano.sh/v1alpha1
kind: ModelBooster
spec:
  backend:
    type: vLLM
    workers:
      - type: Server
        image: vllm/vllm-openai:latest

The ModelBooster is accepted, but the generated ModelServing contains:

workerReplicas: -1

The ModelServing validation webhook rejects this value because workerReplicas cannot be negative. The ModelBooster then remains in Failed status.

Expected Behavior

The configuration should be validated or handled during ModelBooster validation/conversion instead of failing later on the generated ModelServing.

Possible approaches include:

  • Rejecting an invalid or omitted pods value with a clear error pointing to spec.backend.workers[].pods.
  • Handling pods: 0 appropriately so that it never produces a negative workerReplicas.

The important requirement is that a ModelBooster configuration should either be accepted with a valid replica count or be clearly rejected at the ModelBooster level. It should never generate an invalid workerReplicas: -1.

Why This Matters

The current error occurs one level below the resource configured by the user. Instead of clearly indicating that workers[].pods is the problem, the user receives a validation failure for the generated ModelServing.

Validating or handling this value earlier would:

  • Prevent the ModelBooster from entering a persistent Failed state.
  • Avoid generating invalid ModelServing resources.
  • Provide a clear configuration error to the user.
  • Prevent repeated reconciliation attempts for the same invalid configuration.

Proposed Fix

Update the ModelBooster validation or conversion logic to safely handle an omitted or zero workers[].pods value.

The fix should also include regression tests covering:

  • pods omitted from the worker configuration.
  • pods: 0, if explicitly supported by the API.
  • A valid positive pods value.
  • Ensuring workerReplicas is never generated as a negative value.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions