Skip to content

feat: anchor actuation timing on container start (additive) - #1838

Open
rubambiza wants to merge 5 commits into
llm-d:mainfrom
rubambiza:feat/container-start-anchor-pr
Open

feat: anchor actuation timing on container start (additive)#1838
rubambiza wants to merge 5 commits into
llm-d:mainfrom
rubambiza:feat/container-start-anchor-pr

Conversation

@rubambiza

@rubambiza rubambiza commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds a container-start timing anchor alongside the existing pod-create anchor, on both the
standalone (no-FMA) path and the FMA analysis output. Everything here is additive — no existing
column, field, or number changes — so nothing that currently consumes the pod-create anchor breaks.

Closes #1836.

Why

The standalone (no-FMA) baseline is measured from Pod creation (includes scheduling + container
startup), while FMA T_actuation is measured from the requester container's start (excludes that
front-matter). Comparing the two paths across a differing start instant is unsound. This surfaces a
consistent container-start number on both paths so a fair, same-anchor comparison is possible —
while keeping the pod-create numbers in place, since we don't know who else consumes them.

See #1836 for the full rationale; this also addresses the review comment.

Changes

  • workload/harnesses/nop_functions.pyget_container_start() (elapsed Ready − container
    state.running.startedAt, matches the container by name with a sole-container fallback); a
    container_start field on BenchmarkVllmMetrics, captured and assigned next to pod_start.
  • llmdbenchmark/analysis/scripts/nop-analyze_results.py — prints a Container Start(secs) line
    in the vLLM metrics; adds a T_actuation_cstart(s) column (ready − container_start_timestamp) to
    the FMA metrics table (missing anchor renders as --, distinct from a real 0.0).
  • llmdbenchmark/analysis/benchmark_report/native_to_br0_1.py — maps container_start through
    the native->benchmark-report import (the hand-mapped converter otherwise dropped it).
  • Tests for each (15 new tests), including back-compat: older artifacts without container_start
    default gracefully to 0.0 rather than erroring.

Testing

pytest tests/test_nop_container_start.py tests/test_nop_container_start_metrics.py \
  tests/test_nop_analyze_container_start.py tests/test_nop_analyze_fma_cstart.py \
  tests/test_native_to_br01_container_start.py -q
# 15 passed

Validated end-to-end on live standalone Qwen3-4B runs (container_start ~119s) and on existing FMA
4B result artifacts (container_start_timestamp present on every iteration).

Assisted-By: Claude Code

Assisted-By: Claude Code (Anthropic AI) <noreply@anthropic.com>

Signed-off-by: Gloire Rubambiza <gloire@ibm.com>
Assisted-By: Claude Code (Anthropic AI) <noreply@anthropic.com>

Signed-off-by: Gloire Rubambiza <gloire@ibm.com>
Assisted-By: Claude Code (Anthropic AI) <noreply@anthropic.com>

Signed-off-by: Gloire Rubambiza <gloire@ibm.com>
Assisted-By: Claude Code (Anthropic AI) <noreply@anthropic.com>

Signed-off-by: Gloire Rubambiza <gloire@ibm.com>
Assisted-By: Claude Code (Anthropic AI) <noreply@anthropic.com>

Signed-off-by: Gloire Rubambiza <gloire@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Report vLLM container start (not only Pod start) across standalone and FMA baselines

1 participant