Skip to content

Say pipeline-assigned fields are not in system_desc.json #144

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
arav-agarwal2 wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Say pipeline-assigned fields are not in system_desc.json #144

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7785068
Say submission_id is not a system_desc.json field
arav-agarwal2 Oct 9, 2026
f2ae7ff
Cover submission_date and publish_date too
arav-agarwal2 Oct 9, 2026
88053db
Clarify submission ID and date references
arav-agarwal2 Oct 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion endpoints_rules.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2550,6 +2550,20 @@ Endpoints submissions must include the following metadata:
| `config_summary_notes` | Free form field from the submitter to contain information not captured by other fields that concatenate into config_summary. |
| `tps_utilization` | reported_system_tps / (max of all reported_system_tps for all runs) |

`system_desc.json` does not carry `submission_id`, `submission_date`, or `publish_date`. The
submission pipeline assigns all three, so submitters do not set them:

- The submission ID ([§8.5](#85-result-id)) is recorded only as the `[submission_id]/` directory
level ([§8.1](#81-directory-structure)).
- The submission date is set when the bundle is submitted
([Submission Rules §4.1](endpoints_submission_rules.md#41-rolling-submission-model)).
- The publication date is set by the publication cohort or embargo
([Submission Rules §4.2](endpoints_submission_rules.md#42-publication-cohorts-and-embargo)).

Any of these keys left over from an earlier round's template is not a defect: the submission
tooling drops it when it assembles the bundle, and the compliance validator
([§9.1](#91-automated-checks)) does not flag it.

#### 8.2.1 Template Structure

`results/<system>/<model_name>/r<N>/system_desc.json` contains the fields defined in the table above.
Expand Down Expand Up @@ -2662,7 +2676,9 @@ Two identifiers are attached to every submission, and they serve different purpo

The **submission ID** is generated automatically by the submission pipeline as a hash. It is opaque,
carries no meaning, and exists so the lifecycle tooling can track a bundle through upload, review,
and amendment.
and amendment. It is not a submitter-provided field: it appears in the bundle only as the
`[submission_id]/` directory ([§8.1](#81-directory-structure)), never in `system_desc.json`
([§8.2](#82-system-description-system_descjson)).

The **result ID** identifies a single published result and is human-readable. A result is one
published Pareto curve: one system, one benchmark model, one dataset. It is constructed as:
Expand Down
Loading