Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .claude/skills/agent-hub-release/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,8 @@ Per the workflow header (`release_agent_email.yml`), these are **maintainer setu
`baseUrl` (default `https://hub.amd-gaia.ai`).
- **Var `GAIA_HUB_PUBLISH_URL`** — the Worker's **workers.dev** URL for uploads. The
free-plan WAF on the proxied `hub.amd-gaia.ai` custom domain blocks large binary
uploads (but not GETs). Unset → uploads fall back to the custom domain and **403**.
uploads (but not GETs). **Required** — the release fails loudly if it is unset (no
silent fallback to the custom domain).
- **Railway `HUB_CATALOG_URL=https://hub.amd-gaia.ai`** so the website rebuild
reflects the new entry.

Expand Down
23 changes: 19 additions & 4 deletions .github/workflows/release_agent_chat.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,11 +55,13 @@
# restricted to `main` AND the `agent-pkg-*` tag pattern).
# Secrets: GAIA_HUB_TOKEN (Agent Hub Bearer publish token, defined as
# an ENVIRONMENT secret on agent-publish, NOT a repo secret).
# Variables: GAIA_HUB_BASE_URL (default https://hub.amd-gaia.ai),
# Variables: GAIA_HUB_BASE_URL (default https://hub.amd-gaia.ai — downloads),
# GAIA_HUB_PUBLISH_URL (the Worker's workers.dev origin -- large uploads
# are blocked on the proxied custom domain's managed WAF; a wheel is
# small so this matters less than for email's binaries, but the Worker
# endpoint is the same one either way).
# endpoint is the same one either way — same Worker + R2 bucket, just the
# door that isn't WAF-blocked). REQUIRED for publishing — the release
# fails loudly if it is unset (no silent fallback).
#
# Tag namespace: `agent-pkg-chat-*` (NOT `v*` -- avoids firing publish.yml /
# the paused publish_agents.yml). Example:
Expand Down Expand Up @@ -284,6 +286,20 @@ jobs:
exit 1
fi

- name: Assert hub publish URL present
if: steps.dry.outputs.dry_run == 'false'
shell: bash
# Variable goes through env (never inlined into the script text) so a
# value with shell metacharacters can't break parsing.
env:
GAIA_HUB_PUBLISH_URL: ${{ vars.GAIA_HUB_PUBLISH_URL }}
run: |
set -euo pipefail
if [ -z "${GAIA_HUB_PUBLISH_URL:-}" ]; then
echo "::error::missing repository variable GAIA_HUB_PUBLISH_URL — the Worker's workers.dev origin CI POSTs uploads to (the proxied hub.amd-gaia.ai custom domain's managed WAF blocks large uploads). Set it as a repository variable, or as an environment variable on the agent-publish environment. See workers/agent-hub/README.md."
exit 1
fi

- name: Install requests + PyYAML (publish_to_r2.py deps)
if: steps.dry.outputs.dry_run == 'false'
run: python -m pip install --upgrade requests pyyaml
Expand All @@ -296,7 +312,6 @@ jobs:
# the command line or disk.
AGENT_HUB_PUBLISH_TOKEN: ${{ secrets.GAIA_HUB_TOKEN }}
GAIA_HUB_PUBLISH_URL: ${{ vars.GAIA_HUB_PUBLISH_URL }}
GAIA_HUB_BASE_URL: ${{ vars.GAIA_HUB_BASE_URL }}
run: |
set -euo pipefail
whl="$(find dist -maxdepth 1 -name '*.whl' | head -n1)"
Expand All @@ -313,7 +328,7 @@ jobs:
# only special-cases 'email-agent-*'/'.zip' filenames — a bare
# gaia_agent_chat-*.whl has no inferrable prefix.
python hub/agents/email/python/packaging/publish_to_r2.py \
--base-url "${GAIA_HUB_PUBLISH_URL:-${GAIA_HUB_BASE_URL:-https://hub.amd-gaia.ai}}" \
--base-url "${GAIA_HUB_PUBLISH_URL}" \
--manifest "${MANIFEST}" \
--artifact "${whl}=wheel" \
--readme "${README}" \
Expand Down
35 changes: 25 additions & 10 deletions .github/workflows/release_agent_email.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,10 +60,14 @@
# GAIA_HUB_BASE_URL — public Worker origin for downloads + the lock
# baseUrl, default https://hub.amd-gaia.ai.
# GAIA_HUB_PUBLISH_URL — origin CI POSTs uploads to. MUST be the Worker's
# workers.dev URL: the free-plan managed WAF on the
# proxied hub.amd-gaia.ai custom domain blocks large
# binary uploads, but not GET downloads. Unset →
# falls back to GAIA_HUB_BASE_URL (uploads will 403).
# workers.dev URL: the managed WAF fronting the
# hub.amd-gaia.ai custom domain 403s large binary
# uploads, so POST /publish goes through the
# workers.dev origin instead — same Worker + R2
# bucket, just the door that isn't WAF-blocked
# (GET downloads stay on hub.amd-gaia.ai). REQUIRED —
# the release fails loudly if it is unset (no silent
# fallback).
# npm trusted publisher for @amd-gaia/agent-email registered against THIS
# workflow file name: release_agent_email.yml (repo amd/gaia). The OIDC subject
# is tied to the exact filename — renaming this file breaks publish.
Expand Down Expand Up @@ -522,6 +526,19 @@ jobs:
exit 1
fi

- name: Assert hub publish URL present
shell: bash
# Variable goes through env (never inlined into the script text) so a
# value with shell metacharacters can't break parsing.
env:
GAIA_HUB_PUBLISH_URL: ${{ vars.GAIA_HUB_PUBLISH_URL }}
run: |
set -euo pipefail
if [ -z "${GAIA_HUB_PUBLISH_URL:-}" ]; then
echo "::error::missing repository variable GAIA_HUB_PUBLISH_URL — the Worker's workers.dev origin CI POSTs uploads to (the proxied hub.amd-gaia.ai custom domain's managed WAF blocks large uploads). Set it as a repository variable, or as an environment variable on the agent-publish environment. See workers/agent-hub/README.md."
exit 1
fi

- name: Download all platform artifacts
uses: actions/download-artifact@v8
with:
Expand Down Expand Up @@ -593,10 +610,9 @@ jobs:
# blocks large binary multipart uploads (the POST /publish path).
# workers.dev is a separate zone with no such ruleset and hits the same
# Worker + bucket. Downloads and the lock baseUrl stay on
# hub.amd-gaia.ai (GETs aren't blocked). Falls back to the public origin
# if the var is unset.
# hub.amd-gaia.ai (GETs aren't blocked). Required — the release fails
# loudly if the variable is unset.
GAIA_HUB_PUBLISH_URL: ${{ vars.GAIA_HUB_PUBLISH_URL }}
GAIA_HUB_BASE_URL: ${{ vars.GAIA_HUB_BASE_URL }}
run: |
set -euo pipefail
args=()
Expand Down Expand Up @@ -632,7 +648,7 @@ jobs:
capability_matrix_args+=(--capability-matrix "${CAPABILITY_MATRIX}")
fi
python hub/agents/email/python/packaging/publish_to_r2.py \
--base-url "${GAIA_HUB_PUBLISH_URL:-${GAIA_HUB_BASE_URL:-https://hub.amd-gaia.ai}}" \
--base-url "${GAIA_HUB_PUBLISH_URL}" \
--manifest "${MANIFEST}" \
--readme "${README}" \
--changelog "${CHANGELOG}" \
Expand Down Expand Up @@ -692,7 +708,6 @@ jobs:
env:
AGENT_HUB_PUBLISH_TOKEN: ${{ secrets.GAIA_HUB_TOKEN }}
GAIA_HUB_PUBLISH_URL: ${{ vars.GAIA_HUB_PUBLISH_URL }}
GAIA_HUB_BASE_URL: ${{ vars.GAIA_HUB_BASE_URL }}
run: |
set -euo pipefail
VER="${{ steps.ver.outputs.version }}"
Expand All @@ -714,7 +729,7 @@ jobs:
echo "=== package contents ===" && (cd "${STAGE}" && find . -type f | sort)
# Publish the zip as the version's package artifact + its file listing.
python hub/agents/email/python/packaging/publish_to_r2.py \
--base-url "${GAIA_HUB_PUBLISH_URL:-${GAIA_HUB_BASE_URL:-https://hub.amd-gaia.ai}}" \
--base-url "${GAIA_HUB_PUBLISH_URL}" \
--manifest "${MANIFEST}" \
--artifact "agent-email-${VER}.zip=package" \
--package-files package-files.json
Expand Down
27 changes: 22 additions & 5 deletions .github/workflows/release_agent_gaia.yml
Original file line number Diff line number Diff line change
Expand Up @@ -108,10 +108,14 @@
# GAIA_HUB_BASE_URL — public Worker origin for downloads + the lock
# baseUrl. Default https://hub.amd-gaia.ai.
# GAIA_HUB_PUBLISH_URL — origin CI POSTs uploads to. MUST be the Worker's
# workers.dev URL: the free-plan managed WAF on the
# proxied hub.amd-gaia.ai custom domain blocks large
# binary uploads, but not GET downloads. Unset →
# falls back to GAIA_HUB_BASE_URL (uploads 403).
# workers.dev URL: the managed WAF fronting the
# hub.amd-gaia.ai custom domain 403s large binary
# uploads, so POST /publish goes through the
# workers.dev origin instead — same Worker + R2
# bucket, just the door that isn't WAF-blocked
# (GET downloads stay on hub.amd-gaia.ai). REQUIRED —
# the release fails loudly if it is unset (no silent
# fallback).
# npm trusted publisher for @amd-gaia/gaia registered against THIS workflow
# file name: release_agent_gaia.yml (repo amd/gaia). The OIDC subject is tied
# to the exact filename — RENAMING THIS FILE BREAKS THE npm PUBLISH.
Expand Down Expand Up @@ -774,6 +778,19 @@ jobs:
exit 1
fi

- name: Assert hub publish URL present
shell: bash
# Variable goes through env (never inlined into the script text) so a
# value with shell metacharacters can't break parsing.
env:
GAIA_HUB_PUBLISH_URL: ${{ vars.GAIA_HUB_PUBLISH_URL }}
run: |
set -euo pipefail
if [ -z "${GAIA_HUB_PUBLISH_URL:-}" ]; then
echo "::error::missing repository variable GAIA_HUB_PUBLISH_URL — the Worker's workers.dev origin CI POSTs uploads to (the proxied hub.amd-gaia.ai custom domain's managed WAF blocks large uploads). Set it as a repository variable, or as an environment variable on the agent-publish environment. See workers/agent-hub/README.md."
exit 1
fi

- name: Download sidecar artifacts
uses: actions/download-artifact@v8
with:
Expand Down Expand Up @@ -925,7 +942,7 @@ jobs:
# Fail here instead, where the error says to bump the version. Drop
# this flag if the lock is ever switched to --summary-out.
python "${PACKAGING}/publish_to_r2.py" \
--base-url "${GAIA_HUB_PUBLISH_URL:-${GAIA_HUB_BASE_URL:-https://hub.amd-gaia.ai}}" \
--base-url "${GAIA_HUB_PUBLISH_URL}" \
--manifest "${MANIFEST}" \
--strict-immutable \
"${doc_args[@]}" \
Expand Down
33 changes: 26 additions & 7 deletions .github/workflows/release_components.yml
Original file line number Diff line number Diff line change
Expand Up @@ -254,16 +254,15 @@ jobs:
- name: Verify the deployed build is live
env:
GAIA_HUB_PUBLISH_URL: ${{ vars.GAIA_HUB_PUBLISH_URL }}
GAIA_HUB_BASE_URL: ${{ vars.GAIA_HUB_BASE_URL }}
run: |
set -euo pipefail
# No default. Falling back to the custom domain would verify a
# DIFFERENT origin than the publish jobs upload to (they go through
# workers.dev because the WAF fronts hub.amd-gaia.ai), so a silent
# default here reports freshness for something we are not publishing to.
base="${GAIA_HUB_PUBLISH_URL:-${GAIA_HUB_BASE_URL:-}}"
base="${GAIA_HUB_PUBLISH_URL:-}"
if [ -z "${base}" ]; then
echo "::error::neither GAIA_HUB_PUBLISH_URL nor GAIA_HUB_BASE_URL resolved for this job. Both must be REPOSITORY variables — an environment-scoped one resolves empty here, and this check would then target the custom domain rather than the origin the publish jobs POST to."
echo "::error::GAIA_HUB_PUBLISH_URL resolved empty for this job. It must be a REPOSITORY variable — an environment-scoped one resolves empty here, and this check would then target the custom domain rather than the origin the publish jobs POST to."
exit 1
fi
for i in 1 2 3 4 5 6; do
Expand Down Expand Up @@ -369,6 +368,17 @@ jobs:
exit 1
fi

- name: Require the publish URL
if: needs.version.outputs.dry_run == 'false'
env:
GAIA_HUB_PUBLISH_URL: ${{ vars.GAIA_HUB_PUBLISH_URL }}
run: |
set -euo pipefail
if [ -z "${GAIA_HUB_PUBLISH_URL:-}" ]; then
echo "::error::missing repository variable GAIA_HUB_PUBLISH_URL — the Worker's workers.dev origin CI POSTs uploads to (the proxied hub.amd-gaia.ai custom domain's managed WAF blocks large uploads). Set it as a repository variable, or as an environment variable on the agent-publish environment. See workers/agent-hub/README.md."
exit 1
fi

- name: Install publisher deps
if: needs.version.outputs.dry_run == 'false'
run: python -m pip install --upgrade requests pyyaml boto3
Expand All @@ -379,7 +389,6 @@ jobs:
env:
AGENT_HUB_PUBLISH_TOKEN: ${{ secrets.GAIA_HUB_TOKEN }}
GAIA_HUB_PUBLISH_URL: ${{ vars.GAIA_HUB_PUBLISH_URL }}
GAIA_HUB_BASE_URL: ${{ vars.GAIA_HUB_BASE_URL }}
# Artifacts at/over 90 MiB cannot travel through the Worker at all
# (Cloudflare caps request bodies at 100 MB on Free/Pro), so the
# publisher PUTs them straight into R2 and publishes them by
Expand All @@ -392,7 +401,7 @@ jobs:
# Explicit =<platform-key> on every artifact: the publisher's
# inference only special-cases email-agent-* filenames.
python hub/agents/email/python/packaging/publish_to_r2.py \
--base-url "${GAIA_HUB_PUBLISH_URL:-${GAIA_HUB_BASE_URL:-https://hub.amd-gaia.ai}}" \
--base-url "${GAIA_HUB_PUBLISH_URL}" \
--manifest "${TERMINAL_HUB_MANIFEST}" \
--artifact "bin/gaia-linux-x64=linux-x64" \
--artifact "bin/gaia-linux-arm64=linux-arm64" \
Expand Down Expand Up @@ -456,6 +465,17 @@ jobs:
exit 1
fi

- name: Require the publish URL
if: needs.version.outputs.dry_run == 'false'
env:
GAIA_HUB_PUBLISH_URL: ${{ vars.GAIA_HUB_PUBLISH_URL }}
run: |
set -euo pipefail
if [ -z "${GAIA_HUB_PUBLISH_URL:-}" ]; then
echo "::error::missing repository variable GAIA_HUB_PUBLISH_URL — the Worker's workers.dev origin CI POSTs uploads to (the proxied hub.amd-gaia.ai custom domain's managed WAF blocks large uploads). Set it as a repository variable, or as an environment variable on the agent-publish environment. See workers/agent-hub/README.md."
exit 1
fi

- name: Wait for the installer assets on the release
if: needs.version.outputs.dry_run == 'false'
shell: bash
Expand Down Expand Up @@ -512,7 +532,6 @@ jobs:
env:
AGENT_HUB_PUBLISH_TOKEN: ${{ secrets.GAIA_HUB_TOKEN }}
GAIA_HUB_PUBLISH_URL: ${{ vars.GAIA_HUB_PUBLISH_URL }}
GAIA_HUB_BASE_URL: ${{ vars.GAIA_HUB_BASE_URL }}
# Artifacts at/over 90 MiB cannot travel through the Worker at all
# (Cloudflare caps request bodies at 100 MB on Free/Pro), so the
# publisher PUTs them straight into R2 and publishes them by
Expand All @@ -527,7 +546,7 @@ jobs:
# for apt users — publishing it as a second linux-x64 object would
# make the catalog's per-platform download ambiguous.
python hub/agents/email/python/packaging/publish_to_r2.py \
--base-url "${GAIA_HUB_PUBLISH_URL:-${GAIA_HUB_BASE_URL:-https://hub.amd-gaia.ai}}" \
--base-url "${GAIA_HUB_PUBLISH_URL}" \
--manifest "${AGENT_UI_MANIFEST}" \
--artifact "dist/gaia-agent-ui-${VERSION}-x64-setup.exe=win-x64" \
--artifact "dist/gaia-agent-ui-${VERSION}-arm64.dmg=darwin-arm64" \
Expand Down
26 changes: 26 additions & 0 deletions workers/agent-hub/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,32 @@ checked into the repo:
`MAX_ARTIFACT_BYTES` (a plain var, default 250 MiB) caps artifact size and can be
overridden per environment without a secret.

### Publishing origins (one Worker, two doors)

There is exactly **one Agent Hub Worker and one R2 bucket** — `hub.amd-gaia.ai`
and the `workers.dev` URL are two front doors onto the same Worker
(`workers/agent-hub/wrangler.toml`): the custom domain is the user-facing
download door, and the `workers.dev` origin is the CI upload door. The managed
WAF fronting the custom domain **403s large uploads** (the `POST /publish`
path), so CI publishes through the `workers.dev` origin, which has no such
ruleset and hits the same Worker + bucket. A publish through the wrong door
fails loudly at the WAF — it does not land somewhere else.

CI uses two repository variables (set at **repository** level, not environment
level — the version jobs have no `environment:` and an environment-scoped
variable would resolve empty and silently fall back to the hardcode):

| Variable | Value | Purpose |
|---|---|---|
| `GAIA_HUB_BASE_URL` | `https://hub.amd-gaia.ai` | Downloads + the lock `baseUrl` (GETs aren't WAF-blocked) |
| `GAIA_HUB_PUBLISH_URL` | `<worker>.workers.dev` | The origin CI POSTs uploads to. **Required** — the release fails loudly if unset (no silent fallback to a hardcoded URL) |

The publish jobs (`release_agent_*.yml`, `release_components.yml`) now assert
`GAIA_HUB_PUBLISH_URL` is set before publishing, mirroring the existing
`GAIA_HUB_TOKEN` asserts. A missing variable is a startup-time `::error::`
naming what is missing and where to set it, not a 403 halfway through a
release.

## Publishing artifacts larger than 100 MB

A Worker request body is capped by the Cloudflare **account plan** — 100 MB on
Expand Down
Loading