Skip to content

Send point.yaml and system_power.json, with derived power, on runs create - #118

Merged
arav-agarwal2 merged 1 commit into
mainfrom
feat/run-point-and-power
Oct 11, 2026
Merged

arav-agarwal2 merged 1 commit into
mainfrom
feat/run-point-and-power

Conversation

@arav-agarwal2

Copy link
Copy Markdown
Collaborator

Pairs with mlcommons/mlperf-endpoints-api#28, which adds runs.point_config and runs.system_power.

Today only system_desc.json, config.yaml and the result summary reach the database. point.yaml and system_power.json travel only inside the GCS archives, so the visualizer can't see a point's offline mode, its nodes_used, or any power figure.

runs create now sends:

  • point_config: point.yaml, as parsed.

  • system_power: when the run folder has system_power.json, the descriptor unchanged, plus a derived object:

    key meaning
    provisioned_power_kw E.5 total, or null if E.7 rejects the descriptor
    point_power_kw §4.5.3, scaled by the point's nodes_used
    system_tps_per_kw system_tps / point_power_kw, to one decimal (§4.5.2)
    mlc_estimated_power true if any Appendix D default was used, or if the submitter set the flag
    estimated, problems what the computation relied on, and why a figure is null

New submission_checker.power_summary.derive_point_power:

  • It runs SystemPower.compute() and PowerComputation.point_power_kw(), so a stored figure can never disagree with what compliance accepts.
  • It works on parsed contents rather than paths, so the review-repo sync job can import it and recompute when a PR edits these files.
  • It never raises. If a descriptor or nodes_used is something the checker would reject, the affected figure is null and problems says why.
  • The file in the archive is untouched; derived exists only in the database copy.

Compatibility: an API without #28 ignores the new fields, because FastAPI drops unknown body keys, so this can ship in either order.

Testing:

  • New tests/submission_checker/test_power_summary.py covers: fully engaged, scaled by nodes_used, Appendix D default, submitter flag, over-provisioned and unknown-ensemble nodes_used, invalid descriptor, missing throughput, no mutation.
  • Parser tests cover the passthrough and the derived figures against the fixture (14.7 kW).
  • Ruff and mypy are clean. The full suite passes except test_builder.py::test_a_separate_accuracy_run_still_wins, which fails on main too, on local Python 3.10.8 (tarfile.extractall(filter=)); CI uses a newer patch release.
  • Docs: docs/endpoints-cli/usage/runs.md.

🤖 Generated with Claude Code

runs create now sends point.yaml as point_config and, when present,
system_power.json as system_power with a derived object computed by the
checker's own power code: provisioned_power_kw, point_power_kw,
system_tps_per_kw and mlc_estimated_power. The new
submission_checker.power_summary.derive_point_power is importable so the
review-repo sync job computes the same figures.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@arav-agarwal2
arav-agarwal2 merged commit ede8bd3 into main Oct 11, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant