Nightly Cognito integration #106
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: Nightly Cognito integration | |
| # Phase 4e-4 per ADR-0034 §D4 + ROADMAP. Drives the live Cognito User | |
| # Pool (ldz-provisioned per aegis-core#76, available as of 2026-04-24) | |
| # end-to-end against our gateway-side OIDCProvider: | |
| # | |
| # AdminCreateUser → AdminSetUserPassword → AdminInitiateAuth | |
| # → real ID token → OIDCProvider.Authenticate → Principal assertions | |
| # → AdminDeleteUser cleanup | |
| # | |
| # This is the integration test the ADR-0034 §D4 §D1 unit tests' | |
| # httptest mock-JWKS path can't honestly cover — JWKS rotation | |
| # semantics, real Cognito response shapes, and the IRSA → STS → | |
| # Cognito admin-API path all surface here only when run against the | |
| # live pool. | |
| # | |
| # Scheduling per the same logic as nightly post-deploy E2E (see | |
| # postdeploy-e2e.yml): regressions land in the morning operator | |
| # review window, not mid-meeting. workflow_dispatch supports a | |
| # targeted run after a Cognito pool config change or a deliberate | |
| # OIDCProvider edit. | |
| # | |
| # IAM contract (per ldz #76 §B reply 2026-04-24): | |
| # | |
| # Role: arn:aws:iam::251774439261:role/github-actions-aegis-core-cognito-integration | |
| # Trust: repo:BinHsu/aegis-core:ref:refs/heads/main | |
| # Allowed actions: | |
| # cognito-idp:Admin{CreateUser,SetUserPassword,InitiateAuth, | |
| # DeleteUser,GetUser} on Dev User Pool ARN only | |
| # ssm:GetParameter[s] on /aegis/staging/cognito/* | |
| # kms:Decrypt on alias/aegis-staging-secrets | |
| # | |
| # Refs: ADR-0034 §D4, ldz #76 (2026-04-24 reply with role + outputs), | |
| # ROADMAP.md Phase 4e-4. | |
| on: | |
| schedule: | |
| # 03:00 UTC = 11:00 Taipei daylight, before the morning operator | |
| # review. Same cadence as postdeploy-e2e.yml so failures cluster | |
| # at one time-of-day for easier triage. | |
| - cron: "0 3 * * *" | |
| workflow_dispatch: | |
| concurrency: | |
| group: nightly-cognito-integration | |
| cancel-in-progress: false | |
| permissions: | |
| id-token: write # OIDC token exchange to AWS STS | |
| contents: read # checkout | |
| env: | |
| AWS_REGION: eu-central-1 | |
| COGNITO_INTEGRATION_ROLE_ARN: arn:aws:iam::251774439261:role/github-actions-aegis-core-cognito-integration | |
| jobs: | |
| cognito-integration: | |
| name: Live Cognito OIDC integration | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 15 | |
| steps: | |
| - name: Checkout | |
| uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 | |
| # Liveness gate (credential-free): before assuming any AWS role, probe | |
| # the PUBLIC Cognito OIDC discovery document to decide whether staging's | |
| # user pool still exists. No AWS OIDC/IAM/KMS needed. Note the host | |
| # cognito-idp.<region>.amazonaws.com is AWS-owned and always up, so the | |
| # "down" signal here is an HTTP 4xx (pool deleted), NOT a transport | |
| # failure (contrast the gateway /healthz probe in postdeploy-e2e.yml, | |
| # whose host is staging-owned and disappears on teardown): | |
| # - COGNITO_AUTHORITY unset → fork without staging → green skip | |
| # - HTTP 200 → pool exists → assume role and run the integration test | |
| # - HTTP 4xx → pool deleted (staging torn down) → green skip | |
| # - transport failure or 5xx → genuine anomaly → red | |
| # | |
| # This replaces an SSM get-parameter probe: the /aegis/staging/cognito/* | |
| # params are written by a manual operator step (not Terraform), so they | |
| # orphan on teardown and could not reliably signal "staging is down". | |
| - name: Preflight — is staging Cognito up? | |
| id: preflight | |
| env: | |
| COGNITO_AUTHORITY: ${{ vars.COGNITO_AUTHORITY }} | |
| run: | | |
| set -uo pipefail | |
| # GitHub's default step shell is `bash -e {0}`, so errexit is ON even | |
| # though we never set it. Turn it OFF here: a failing `HTTP=$(curl …)` | |
| # would otherwise kill the step on the first attempt — defeating the | |
| # retry loop and the transport-failure classification below. We read | |
| # curl's exit code explicitly instead of letting -e act on it. | |
| set +e | |
| if [ -z "${COGNITO_AUTHORITY:-}" ]; then | |
| echo "staging_up=false" >> "$GITHUB_OUTPUT" | |
| echo "::notice title=Nightly Cognito integration skipped::vars.COGNITO_AUTHORITY unset — no staging Cognito pool configured (fork-friendly default). Skipping." | |
| exit 0 | |
| fi | |
| URL="${COGNITO_AUTHORITY%/}/.well-known/openid-configuration" | |
| # -sS: quiet but show transport errors; no -f, so we read the status | |
| # code ourselves and classify down (4xx) vs broken (5xx/transport). | |
| # | |
| # Retry transport failures (RC != 0) up to 3 attempts before treating | |
| # them as a hard anomaly: this host is AWS-owned and always up, so a | |
| # transport failure is rare — but a single transient runner/DNS blip | |
| # must not red the nightly. Any HTTP response ends the loop at once. | |
| HTTP=000; RC=0 | |
| for attempt in 1 2 3; do | |
| HTTP=$(curl -sS -o /dev/null -w '%{http_code}' --max-time 10 "$URL" 2>/dev/null) | |
| RC=$? | |
| [ "$RC" -eq 0 ] && break | |
| [ "$attempt" -lt 3 ] && { echo "Probe attempt ${attempt} failed (curl exit ${RC}); retrying…"; sleep 5; } | |
| done | |
| if [ "$RC" -ne 0 ]; then | |
| echo "::error title=Cognito OIDC endpoint unreachable::Could not reach ${URL} after 3 attempts (curl exit ${RC}). cognito-idp.<region>.amazonaws.com is normally always up, so this is a real network/AWS anomaly — failing." >&2 | |
| exit 1 | |
| elif [ "$HTTP" = "200" ]; then | |
| echo "staging_up=true" >> "$GITHUB_OUTPUT" | |
| echo "Cognito OIDC discovery at ${URL} returned 200 — pool is up; running integration test." | |
| elif [ "$HTTP" -ge 400 ] && [ "$HTTP" -lt 500 ]; then | |
| echo "staging_up=false" >> "$GITHUB_OUTPUT" | |
| echo "::notice title=Nightly Cognito integration skipped::Cognito OIDC discovery returned HTTP ${HTTP} — staging user pool is gone (torn down). Skipping. Bring staging up (and refresh vars.COGNITO_AUTHORITY) to re-enable." | |
| else | |
| echo "::error title=Cognito OIDC endpoint error::${URL} returned HTTP ${HTTP} (expected 200 or 4xx). Unexpected server-side error — failing so it is visible." >&2 | |
| exit 1 | |
| fi | |
| # AWS OIDC: STS AssumeRoleWithWebIdentity → temporary creds. Only after | |
| # the credential-free liveness gate confirms the pool is up. The | |
| # configure-aws-credentials action exports them as | |
| # AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY / AWS_SESSION_TOKEN | |
| # env vars; bazel test forwards them via env_inherit in | |
| # gateway_go/internal/auth/BUILD.bazel. | |
| - name: Configure AWS credentials (OIDC) | |
| if: steps.preflight.outputs.staging_up == 'true' | |
| uses: aws-actions/configure-aws-credentials@517a711dbcd0e402f90c77e7e2f81e849156e31d | |
| with: | |
| aws-region: ${{ env.AWS_REGION }} | |
| role-to-assume: ${{ env.COGNITO_INTEGRATION_ROLE_ARN }} | |
| role-session-name: aegis-core-cognito-integration-${{ github.run_id }} | |
| # Read live values from SSM PS rather than hard-coding the | |
| # Cognito IDs into the workflow. Two reasons per ldz #76 §A: | |
| # (1) survives any baking-into-Secrets drift; (2) the SSM PS | |
| # values are the canonical source — the workflow file becomes | |
| # the wrong place for them to live if they ever rotate. | |
| - name: Read Cognito IDs from SSM Parameter Store | |
| id: ssm | |
| if: steps.preflight.outputs.staging_up == 'true' | |
| run: | | |
| set -euo pipefail | |
| # The Cognito SSM parameters are stored as `SecureString` (KMS | |
| # encrypted under alias/aegis-staging-secrets — see the IAM | |
| # contract above). Without `--with-decryption` the CLI returns | |
| # the encrypted ciphertext, which then sails through every | |
| # downstream check (it is, technically, a non-empty string) | |
| # and only fails much later when AWS rejects the ciphertext as | |
| # a User Pool ID — surfacing as a misleading | |
| # `AccessDeniedException` against a ciphertext-shaped resource | |
| # ARN. Decrypt at read time so all downstream callers see | |
| # plaintext. | |
| # | |
| # Capture into local vars first so `set -e` actually propagates | |
| # the AWS CLI exit code (it does NOT propagate cleanly out of | |
| # `echo "X=$(aws ssm ...)" >> $GITHUB_ENV` — `>> $GITHUB_ENV` | |
| # also doesn't set the current shell, which is the second half | |
| # of the trap). Targeted annotation on read failure points | |
| # operators at the most likely buckets without prejudging which | |
| # one fired. | |
| read_param() { | |
| local name=$1 | |
| local value | |
| if ! value=$(aws ssm get-parameter \ | |
| --name "$name" \ | |
| --with-decryption \ | |
| --query 'Parameter.Value' \ | |
| --output text 2>&1); then | |
| echo "::error title=Cognito SSM PS read failed::Parameter ${name} not readable. Likely buckets: (1) parameter destroyed (check ldz teardown logs); (2) KMS Decrypt denied on alias/aegis-staging-secrets (check IAM policy on the integration role); (3) STS role assumption returned creds without ssm:GetParameter[s] on this path. Raw CLI output is in the step log above this annotation." | |
| echo "aws ssm get-parameter raw output:" >&2 | |
| echo "${value}" >&2 | |
| exit 1 | |
| fi | |
| printf '%s' "$value" | |
| } | |
| POOL_ID=$(read_param /aegis/staging/cognito/user-pool-id) | |
| CLIENT_ID=$(read_param /aegis/staging/cognito/app-client-id) | |
| ISSUER=$(read_param /aegis/staging/cognito/issuer-url) | |
| echo "::add-mask::$CLIENT_ID" | |
| { | |
| echo "AEGIS_COGNITO_USER_POOL_ID=$POOL_ID" | |
| echo "AEGIS_COGNITO_APP_CLIENT_ID=$CLIENT_ID" | |
| echo "AEGIS_COGNITO_ISSUER_URL=$ISSUER" | |
| } >> "$GITHUB_ENV" | |
| echo "Cognito IDs loaded into env for bazel test." | |
| # Bazel test target. env_inherit on auth_test forwards all the | |
| # AWS_* + AEGIS_COGNITO_* env vars into the sandboxed test | |
| # binary. The unit tests (httptest mock JWKS, claim-mapping, | |
| # error paths) still run alongside the integration test — | |
| # their cost is sub-second and runs as a free regression check | |
| # on the same nightly cadence. | |
| - name: Cache Bazel | |
| if: steps.preflight.outputs.staging_up == 'true' | |
| uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5 | |
| with: | |
| path: | | |
| ${{ github.workspace }}/.bazel_cache | |
| ${{ github.workspace }}/.bazelisk | |
| key: bazel-cognito-integration-${{ runner.os }}-${{ hashFiles('MODULE.bazel', 'MODULE.bazel.lock', '.bazelversion', '.bazelrc') }} | |
| restore-keys: | | |
| bazel-cognito-integration-${{ runner.os }}- | |
| - name: Run auth integration test against live Cognito | |
| if: steps.preflight.outputs.staging_up == 'true' | |
| run: | | |
| set -euo pipefail | |
| ./tools/bazelisk/bazelisk test \ | |
| //gateway_go/internal/auth:auth_test \ | |
| --test_output=errors \ | |
| --test_filter=TestOIDCIntegrationCognito \ | |
| --nocache_test_results | |
| # If the test failed, surface a workflow annotation pointing at | |
| # the most likely causes — operator triaging the morning | |
| # alert shouldn't have to re-derive the failure-mode taxonomy. | |
| # Guard with staging_up so this hint only fires for real | |
| # test failures, not for the clean "staging is down" skip path. | |
| - name: Failure triage hint | |
| if: failure() && steps.preflight.outputs.staging_up == 'true' | |
| run: | | |
| cat <<'EOF' >&2 | |
| ::error title=Cognito integration failed:: | |
| Most likely causes (in declining order of historical | |
| frequency on similar nightly integrations): | |
| 1. AWS STS role assumption failed → check ldz IAM role | |
| github-actions-aegis-core-cognito-integration trust | |
| policy still scopes refs/heads/main + this run's sub. | |
| 2. Cognito User Pool deletion / rotation per Runbook 008 | |
| → SSM PS values stale. Re-run baseline on ldz side | |
| then re-trigger this workflow. | |
| 3. JWKS endpoint unreachable / signature mismatch → | |
| Cognito-side regional outage; check AWS health | |
| dashboard for eu-central-1 cognito-idp. | |
| 4. Test-user creation throttled → AdminCreateUser has a | |
| soft throttle. Re-run; if persistent, raise an ldz | |
| cross-repo issue for quota review. | |
| EOF |