From 778506802fc9ef8f69e73e83982e0e459de9807a Mon Sep 17 00:00:00 2001 From: Arav Agarwal <43820556+arav-agarwal2@users.noreply.github.com> Date: Fri, 9 Oct 2026 17:07:40 -0400 Subject: [PATCH 1/3] Say submission_id is not a system_desc.json field The submission ID is assigned by the pipeline (8.5) and lives only in the / directory (8.1). Say so in 8.2 and 8.5, and say a key left over from a v0.7 template is dropped by the tooling and not flagged. Fixes #143. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01N2grVghMMtp7LPurYJ5o35 --- endpoints_rules.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/endpoints_rules.md b/endpoints_rules.md index dfd4ed5..415adb9 100644 --- a/endpoints_rules.md +++ b/endpoints_rules.md @@ -2550,6 +2550,13 @@ Endpoints submissions must include the following metadata: | `config_summary_notes` | Free form field from the submitter to contain information not captured by other fields that concatenate into config_summary. | | `tps_utilization` | reported_system_tps / (max of all reported_system_tps for all runs) | +`system_desc.json` does not carry a `submission_id`. The submission ID is assigned by the +submission pipeline ([§8.5](#85-result-id)) and is recorded only as the `[submission_id]/` +directory level ([§8.1](#81-directory-structure)), so submitters do not set it. A `submission_id` +key left over from an earlier round's template is not a defect: the submission tooling drops it +when it assembles the bundle, and the compliance validator ([§9.1](#91-automated-checks)) does not +flag it. + #### 8.2.1 Template Structure `results///r/system_desc.json` contains the fields defined in the table above. @@ -2662,7 +2669,9 @@ Two identifiers are attached to every submission, and they serve different purpo The **submission ID** is generated automatically by the submission pipeline as a hash. It is opaque, carries no meaning, and exists so the lifecycle tooling can track a bundle through upload, review, -and amendment. +and amendment. It is not a submitter-provided field: it appears in the bundle only as the +`[submission_id]/` directory ([§8.1](#81-directory-structure)), never in `system_desc.json` +([§8.2](#82-system-description-system_descjson)). The **result ID** identifies a single published result and is human-readable. A result is one published Pareto curve: one system, one benchmark model, one dataset. It is constructed as: From f2ae7ffb75b52d7ac81c8651e3c01f8550a0c603 Mon Sep 17 00:00:00 2001 From: Arav Agarwal <43820556+arav-agarwal2@users.noreply.github.com> Date: Fri, 9 Oct 2026 17:21:22 -0400 Subject: [PATCH 2/3] Cover submission_date and publish_date too Like the submission ID, the pipeline assigns the submission date (Submission Rules 5.2) and the publication date (Submission Rules 4.2), and v0.7 templates carried both. Extend the 8.2 note to all three. Part of #143. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01N2grVghMMtp7LPurYJ5o35 --- endpoints_rules.md | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/endpoints_rules.md b/endpoints_rules.md index 415adb9..5aaf4fb 100644 --- a/endpoints_rules.md +++ b/endpoints_rules.md @@ -2550,12 +2550,19 @@ Endpoints submissions must include the following metadata: | `config_summary_notes` | Free form field from the submitter to contain information not captured by other fields that concatenate into config_summary. | | `tps_utilization` | reported_system_tps / (max of all reported_system_tps for all runs) | -`system_desc.json` does not carry a `submission_id`. The submission ID is assigned by the -submission pipeline ([§8.5](#85-result-id)) and is recorded only as the `[submission_id]/` -directory level ([§8.1](#81-directory-structure)), so submitters do not set it. A `submission_id` -key left over from an earlier round's template is not a defect: the submission tooling drops it -when it assembles the bundle, and the compliance validator ([§9.1](#91-automated-checks)) does not -flag it. +`system_desc.json` does not carry `submission_id`, `submission_date`, or `publish_date`. The +submission pipeline assigns all three, so submitters do not set them: + +- The submission ID ([§8.5](#85-result-id)) is recorded only as the `[submission_id]/` directory + level ([§8.1](#81-directory-structure)). +- The submission date is set when the bundle is submitted + ([Submission Rules §5.2](endpoints_submission_rules.md#52-how-to-submit)). +- The publication date is set by the publication cohort or embargo + ([Submission Rules §4.2](endpoints_submission_rules.md#42-publication-cohorts-and-embargo)). + +Any of these keys left over from an earlier round's template is not a defect: the submission +tooling drops it when it assembles the bundle, and the compliance validator +([§9.1](#91-automated-checks)) does not flag it. #### 8.2.1 Template Structure From 88053dbd85bf96a2108c12bda42abd81e7a6a33b Mon Sep 17 00:00:00 2001 From: Arav Agarwal <43820556+arav-agarwal2@users.noreply.github.com> Date: Sun, 11 Oct 2026 14:35:49 -0400 Subject: [PATCH 3/3] Clarify submission ID and date references Updated references to submission rules and clarified submission ID recording. Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --- endpoints_rules.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/endpoints_rules.md b/endpoints_rules.md index 5aaf4fb..b0eb21b 100644 --- a/endpoints_rules.md +++ b/endpoints_rules.md @@ -2556,7 +2556,7 @@ submission pipeline assigns all three, so submitters do not set them: - The submission ID ([§8.5](#85-result-id)) is recorded only as the `[submission_id]/` directory level ([§8.1](#81-directory-structure)). - The submission date is set when the bundle is submitted - ([Submission Rules §5.2](endpoints_submission_rules.md#52-how-to-submit)). + ([Submission Rules §4.1](endpoints_submission_rules.md#41-rolling-submission-model)). - The publication date is set by the publication cohort or embargo ([Submission Rules §4.2](endpoints_submission_rules.md#42-publication-cohorts-and-embargo)).