Skip to content

ci(hub): one approval per component release, not two - #3018

Merged
kovtcharov-amd merged 3 commits into
mainfrom
ci/single-approval-gate
Aug 20, 2026
Merged

ci(hub): one approval per component release, not two#3018
kovtcharov-amd merged 3 commits into
mainfrom
ci/single-approval-gate

Conversation

@kovtcharov-amd

Copy link
Copy Markdown
Collaborator

Why this matters

Releasing hub components asks for approval twice. GitHub raises one prompt per batch of jobs that become pending together — terminal-hub and agent-ui pend simultaneously and share one, but deploy-worker has to finish before them, so it always forms an earlier batch of its own. Any environment-gated job placed before the publishes does this; it was a side effect of #2991 rather than a deliberate choice.

Dropping the environment from deploy-worker leaves exactly one gate, on the step that actually needs a human.

Test plan

  • Move CLOUDFLARE_API_TOKEN and CLOUDFLARE_ACCOUNT_ID to repository secrets (they are currently on the agent-publish environment, where this job can no longer read them)
  • Dispatch with dry_run=false — exactly one approval prompt, raised after deploy-worker has already run
  • Confirm deploy-worker still gates the publishes: they must not start if it fails
  • Dispatch with dry_run=true — no deploy, no approval, worker-check still runs
🔍 The trade-off

The reviewer gate exists for the part that cannot be undone: an immutable R2 upload. The Worker deploy is neither irreversible (redeploying fixes it) nor unchecked — worker-check runs typecheck, the vitest suite, and a full bundle first, and the deploy only happens on a real release someone dispatched.

The genuine cost is secret scope: repository secrets are readable by every workflow in the repo, where environment secrets were readable by one. That is a real widening and the reason to keep the token minimally scoped — Workers + R2 on the single account, plus the amd-gaia.ai zone for the custom-domain route.

The alternative that keeps them environment-scoped is a second environment with the same reviewers, which trades the extra prompt for an extra environment to keep in sync. Happy to go that way instead if the wider secret scope is the bigger concern.

Putting deploy-worker in the agent-publish environment cost a second approval
prompt. GitHub raises one per batch of jobs that pend together: terminal-hub
and agent-ui pend simultaneously and share a prompt, but the deploy has to
finish before them, so it always lands in an earlier batch of its own.

Dropped the environment from deploy-worker. The reviewer gate is there for the
step that cannot be undone — an immutable R2 upload — and this deploy is
neither irreversible (redeploying fixes it) nor unchecked (worker-check runs
typecheck, the vitest suite, and a bundle before it). It also only runs on a
real release someone dispatched, never on a dry run, and the publishes still
refuse to start unless it succeeded.

This moves CLOUDFLARE_API_TOKEN and CLOUDFLARE_ACCOUNT_ID from environment
secrets to repository secrets, which widens their reach from one environment to
every workflow in the repo. That is the real cost of the change; the error
message and README now say which kind is required so a half-migrated setup
fails by name rather than inside wrangler.
@github-actions github-actions Bot added the devops DevOps/infrastructure changes label Aug 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Verdict: Request changes — one settings change away from approvable.

This drops the reviewer gate from the job that deploys the Agent Hub Worker so a component release only prompts a maintainer once instead of twice. The goal is right and the reasoning is written down where the next maintainer will find it, but the change does two things beyond removing a prompt, and neither is accounted for in the comment.

🔒 SECURITY CONCERN: this widens who can push code to the production Hub Worker, and where its Cloudflare credentials live. @kovtcharov-amd

  • The Cloudflare API token (Workers edit + R2) has to become a repository secret. Today it is only readable after a human approves the gate; afterwards it is readable by every workflow in the repo.
  • The deploy job also loses the environment's allowed-branches/tags rule — not just the approval. Anyone with write access can now run this workflow from an arbitrary branch with the dry-run box unchecked and push that branch's Worker code straight to production, unreviewed. The comment's claim that it "only runs on a real release the maintainer dispatched" no longer holds.

Both are avoidable while still getting one approval: put the deploy job in a second environment that has no required reviewers but keeps the branch/tag restriction, and hold the Cloudflare secrets there. No extra prompt, credentials stay environment-scoped, and the release can still only deploy from a real release ref.

Two smaller things: the freshness check that runs right after the deploy reads a repository variable that, until now, was only ever read from inside the gated environment — worth confirming it is defined repo-wide, or that step will quietly check the wrong origin. And this PR is not self-contained: the secrets must be re-created at repo level before the next release tag, or the whole component release fails.

Real-world evidence

N/A — CI workflow and docs only; no evidence bundle was produced for this PR, and this code path can only run during an actual tagged release. The verdict rests on static review of the workflow. (gh was unavailable in this review environment, so I could not read the PR description — if it already records the required repo-settings migration, disregard the last nit below.)

🔍 Technical details

Issues

🔴 Deploy job loses the environment's ref allowlist, not just the approval (.github/workflows/release_components.yml:174-178)

agent-publish carries a deployment tag allowlist in repo settings — the workflow says so itself at line 246-249 ("it must include v* or every release tag is rejected"). Removing environment: removes that restriction along with the reviewer. Combined with on.workflow_dispatch (dispatchable from any branch containing the file) and if: dry_run == 'false', any write-access account can deploy arbitrary Worker code to production. The Worker is the manifest validator and holds the R2 binding for the hub catalog, so this is a supply-chain-adjacent surface, and "redeploy fixes it" understates a malicious deploy rather than a bad one.

Recommended shape — a non-gated environment that keeps the ref rule:

    runs-on: ubuntu-latest
    # Separate environment from agent-publish on purpose: no required reviewers,
    # so it does not add a second approval prompt, but it keeps the deployment
    # branch/tag allowlist and scopes CLOUDFLARE_* out of repo-wide secrets.
    environment: worker-deploy

(Requires creating worker-deploy with the same v* + main allowlist as agent-publish, reviewers left empty, and the two CLOUDFLARE_* secrets moved there rather than to repo scope. If that is done, the ::error:: text at line 202 should say "environment secret(s) on worker-deploy" instead of "repository secret(s)".)

🟡 vars.GAIA_HUB_PUBLISH_URL is now read outside agent-publish for the first time (.github/workflows/release_components.yml:215-219)

Every other reference to GAIA_HUB_PUBLISH_URL across release_agent_gaia.yml, release_agent_email.yml, release_agent_chat.yml, and this file sits in a job with environment: agent-publish. If it is an environment variable, this step now resolves it empty and falls through to https://hub.amd-gaia.ai — the custom domain, when the step's own comment (lines 210-212) explains that workers.dev is "the origin whose freshness matters" because the WAF sits in front of the custom domain. .claude/skills/agent-hub-release/SKILL.md:176-180 lists it as a "Var" without stating the scope, so this is not resolvable from the tree.

Confirm it is repo-level. Separately, per CLAUDE.md's no-silent-fallbacks rule, this step would be better failing loudly than silently verifying a different origin than the publish jobs upload to:

          base="${GAIA_HUB_PUBLISH_URL:-${GAIA_HUB_BASE_URL:-}}"
          if [ -z "${base}" ]; then
            echo "::error::neither GAIA_HUB_PUBLISH_URL nor GAIA_HUB_BASE_URL is set for this job. Both must be REPOSITORY variables — this job runs outside agent-publish, so environment-scoped vars resolve empty here and the freshness check would silently target the custom domain instead of the workers.dev origin the publish jobs POST to."
            exit 1
          fi

🟢 The comment overstates the trigger surface (.github/workflows/release_components.yml:170-171)

"It also only runs on a real release the maintainer dispatched, never on a dry run" — the workflow also fires on any v* tag push, and "dispatched" covers anyone with write access, not just a maintainer. Worth tightening so the next reader does not inherit the assumption.

🟢 Migration is not self-contained

The secrets must exist at repo level before the next v* tag or deploy-worker fails and blocks both publish jobs (it does fail loudly with an actionable message — that part is right). Worth stating in the PR description, and worth deleting the now-shadowed agent-publish-scoped copies afterwards so there is one source of truth.

Strengths

  • The fail-loud secret precondition was updated in lockstep with the scope change, and the error message still names what is missing, what to do, and where to read more — it did not decay into a generic "missing secret".
  • workers/agent-hub/README.md:255-256 was corrected in the same PR, so the setup instructions cannot ship contradicting the workflow. That is the doc-sync rule being honored without being asked.
  • The rationale lives in the workflow comment rather than only in the commit message, which is where the next person debugging an approval prompt will actually look.

The review caught that dropping `environment:` threw away more than the second
approval. agent-publish also carries the deployment ref allowlist — main, v*,
agent-pkg-* — so without it anyone with write access could dispatch this
workflow from an arbitrary branch with dry_run unchecked and push that branch's
Worker straight to production. The Worker is the manifest validator and holds
the R2 binding, so that is a supply-chain surface, and "redeploy fixes it"
answers a bad deploy rather than a malicious one. It also moved the Cloudflare
credentials from one environment to every workflow in the repo.

deploy-worker now runs in a new worker-deploy environment: no required
reviewers, so the release still prompts once, but the same ref allowlist as
agent-publish and the Cloudflare secrets scoped to it.

Also fixed the freshness check, which defaulted to hub.amd-gaia.ai when neither
hub URL variable resolved. The publish jobs POST to workers.dev — the WAF fronts
the custom domain — so that default verified an origin we do not upload to. It
now fails loudly instead. (GAIA_HUB_PUBLISH_URL is in fact repo-level, so this
was latent rather than live, but a silent wrong-origin check is exactly what the
no-silent-fallbacks rule is for.)
@kovtcharov-amd

Copy link
Copy Markdown
Collaborator Author

Fixed — and the 🔴 was a genuine hole I introduced, not just a scoping preference.

You're right that removing environment: discards the deployment ref allowlist along with the reviewer. I had reasoned about the approval and not noticed the second thing the environment was doing, which left any write-access account able to dispatch from an arbitrary branch with dry-run unchecked and deploy that branch's Worker to production. "Redeploy fixes it" answers a bad deploy; it doesn't answer a malicious one.

Took your recommended shape. worker-deploy now exists with no required reviewers and the same ref allowlist as agent-publish — mirrored exactly:

Environment Reviewers Allowed refs
agent-publish required main, v*, agent-pkg-*
worker-deploy none main, v*, agent-pkg-*

One prompt, credentials stay environment-scoped, and the deploy can still only run from a real release ref.

🔍 On the two smaller points

GAIA_HUB_PUBLISH_URL is repo-level — I checked repos/amd/gaia/actions/variables and it's there, with nothing of that name on agent-publish. So the wrong-origin failure was latent rather than live. I took the fix anyway: the step no longer defaults to hub.amd-gaia.ai and fails loudly if neither variable resolves, since silently verifying a different origin than the publish jobs upload to is precisely the shape the no-silent-fallbacks rule exists to prevent.

Migration. The Cloudflare secrets now need to live on worker-deploy rather than at repo level. I've created the environment and its policies; only the secret values need adding, which I can't do. Noted in the PR description and the README table.

Trigger surface. The comment no longer claims "only a real release the maintainer dispatched" — it now says what actually constrains the job, which is the ref allowlist.

One thing to be aware of: I created the worker-deploy environment and its three branch/tag policies via the API so the config exists before merge. If you'd rather own repo settings by hand, it's trivially deletable and I'll adjust.

@github-actions

Copy link
Copy Markdown
Contributor

Verdict: Approve

This is a clean CI security improvement. Moving deploy-worker from agent-publish to a new worker-deploy environment (no required reviewers, same ref allowlist) eliminates the double-approval UX problem while preserving the ref restriction that prevents arbitrary-branch Worker pushes. The fail-loud URL check is a correct application of the "no silent fallbacks" principle — the previous :-https://hub.amd-gaia.ai default silently verified the wrong origin.

One suggestion before this goes live operationally:

🟢 The README documents the two secrets the worker-deploy environment needs but doesn't say what ref allowlist to configure when creating the environment. The security guarantee of this change ("same ref allowlist as agent-publish") is only true if the person setting up worker-deploy knows to add main / v* / agent-pkg-* as deployment branch rules — that detail lives only in the YAML comment, not in the setup instructions.

🔍 Technical details

workers/agent-hub/README.md:255-257 documents the secrets but omits the ref allowlist setup step. A one-liner addition after the secret table would make the setup self-contained:

Set the **deployment branch and tag rules** on `worker-deploy` to match `agent-publish`:
`main`, `v*`, `agent-pkg-*`. Leave **Required reviewers** empty.

The deploy-worker job at release_components.yml:178 is otherwise correct: needs: [version, worker-check] ensures it runs before any publish job, and the downstream publish if: conditions already handle the skipped case cleanly.

…n it

The setup instructions listed the two secrets but not the environment settings,
so the security property this change rests on — the deployment ref allowlist —
existed only as a comment in the YAML. Anyone creating worker-deploy from the
README would have got an environment with no reviewers and no ref restriction,
which is strictly worse than the agent-publish arrangement it replaced.

Both settings are now documented with why they matter: reviewers empty (the
second prompt is the thing being avoided) and main / v* / agent-pkg-* (the only
remaining restriction on a job that deploys the manifest validator).
@kovtcharov-amd

Copy link
Copy Markdown
Collaborator Author

Good catch — fixed in the latest push.

You're right that the guarantee was only as good as the setup instructions, and those lived in a YAML comment rather than where someone creating the environment would look. A worker-deploy built from the README as written would have had no reviewers and no ref restriction, which is strictly worse than the agent-publish arrangement it replaces.

The README now documents both settings with the reason each matters — reviewers left empty because the second prompt is the whole point, and main / v* / agent-pkg-* because it is the only remaining restriction on a job that deploys the manifest validator.

For what it's worth the live environment already matches; I created it and its three policies via the API while addressing the earlier round, so the config and the docs now agree.

@github-actions

Copy link
Copy Markdown
Contributor

Verdict: Approve with suggestions

The environment split is well-reasoned and the fail-loud base URL change is a clear CLAUDE.md compliance win. One security trade-off is worth a maintainer's explicit sign-off.

🟡 The worker-deploy environment intentionally has no required reviewers, meaning any contributor with write access to main, v*, or agent-pkg-* can now deploy the Agent Hub Worker to production without human approval — they just dispatch the workflow and let the worker-deploy job complete, then cancel before approving agent-publish. Previously the Worker deploy lived in agent-publish, so the same human gate covered both. The PR documents this trade-off clearly in the YAML comment and README, but if this is intentional a maintainer should confirm it before merge.

Everything else looks clean:

  • Error messages consistently updated to reference worker-deploy.
  • Base URL fail-loud path is correct: removing the silent fallback to hub.amd-gaia.ai prevents the health check from silently verifying the wrong origin.
  • README environment setup table is complete and the "easy to miss" note on the ref allowlist is genuinely useful.
🔍 Technical details

Security trade-off (release_components.yml:187)

Old flow: deploy-workerenvironment: agent-publish → reviewer required → Worker deployed.

New flow: deploy-workerenvironment: worker-deploy → no reviewer → Worker deployed immediately. The agent-publish gate only covers the downstream publish jobs.

Attack surface: write access + push to agent-pkg-* + workflow dispatch with dry_run: false → Worker (manifest validator, R2 binding) deployed without approval. The ref allowlist is the only remaining guard.

This is a real change in the security boundary, even if the threat model (trusted contributors with write access) makes it acceptable. Worth a maintainer's explicit "yes, this is intended" before merge.

Base URL check (release_components.yml:221-229)

The new explicit [ -z "${base}" ] guard is correct. The previous silent fallback to the custom domain (https://hub.amd-gaia.ai) would have verified the wrong origin — good catch.

The error message note ("an environment-scoped one resolves empty here") is accurate: GAIA_HUB_PUBLISH_URL on agent-publish would be empty in the worker-deploy job. Both vars must be repository-scoped.

@kovtcharov-amd
kovtcharov-amd added this pull request to the merge queue Aug 20, 2026
Merged via the queue into main with commit b254708 Aug 20, 2026
28 checks passed
@kovtcharov-amd
kovtcharov-amd deleted the ci/single-approval-gate branch August 20, 2026 21:40
dislovelhl pushed a commit to dislovelhl/amd-gaia that referenced this pull request Aug 21, 2026
…rker body cap (amd#3025)

## Why this matters

The Agent UI installers have never been publishable to the hub, and the
error blamed the wrong thing. Each one is 106–135 MiB, and **Cloudflare
caps a Worker request body at 100 MB on Free/Pro**, so they are rejected
with a `413` by the edge before the Worker executes.
`MAX_ARTIFACT_BYTES` (250 MiB) was never consulted — the HTML error body
is Cloudflare's, not ours — so raising it would have changed nothing.
email and terminal-hub only work because they are small (43.5 MiB and
under).

Artifacts at or above 90 MiB now go straight into the same R2 bucket
over the S3 API, which has no such cap, and are published **by
reference**: the POST carries the artifact's coordinates instead of its
bytes.

## Test plan

- [ ] Add `R2_ACCESS_KEY_ID` / `R2_SECRET_ACCESS_KEY` repo secrets
(Cloudflare → R2 → Manage API Tokens → Object Read & Write on
`gaia-hub`)
- [ ] `cd workers/agent-hub && npm test` — 214 pass, 10 of them new
- [ ] `pytest
hub/agents/email/python/tests/test_publish_to_r2_by_reference.py` — 7
pass
- [ ] Dispatch `release_components.yml` with `dry_run=false` and confirm
**agent-ui publishes** where it previously 413'd
- [ ] Confirm terminal-hub and email still take the inline path
unchanged

<details>
<summary>🔍 How the integrity guarantee survives</summary>

The Worker no longer sees these bytes, so "the publisher said so" would
be an easy accidental outcome. It isn't: before recording anything, the
Worker heads the object and checks its size **and** SHA-256 against what
R2 itself stored at PUT time.

R2 keeps a whole-object SHA-256 only for **single-part** uploads. An
object without one is refused (`artifact_unverifiable`) rather than
trusted — so the uploader uses `put_object` with `ChecksumSHA256`, never
`upload_file`, which would switch to multipart and silently drop the
checksum. A test pins that call shape, because the failure mode is a
green upload followed by a rejected publish, mid-release.

**Immutability needed rethinking rather than reusing.** Inline, the
object's presence in R2 *is* the record, so heading it is the right
check. By reference the object always exists by the time the Worker is
called, so that same check would 409 against the caller's own upload —
the record is the agent manifest, which only lists artifacts this
endpoint accepted.

Both new-path tests were verified non-vacuous by mutating the verifier
to trust the caller's claim; the hash-mismatch and missing-checksum
cases fail as they should.

**New secrets are R2 S3 credentials**, distinct from the
`CLOUDFLARE_API_TOKEN` that deploys the Worker. Missing any of the three
fails loudly naming all of them — never a fallback to the Worker path,
which cannot work at these sizes.
</details>

Stacked on amd#2991 (merged) and independent of amd#3018.

---------

Co-authored-by: Ovtcharov <kovtchar@amd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

devops DevOps/infrastructure changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant