Add first-class AWS CodeArtifact publishing and consumption via GitHub OIDC

[markrichardson]

Problem / motivation

Rhiza supports public PyPI publishing through trusted publishing and custom package feeds through PYPI_REPOSITORY_URL plus a static PYPI_TOKEN. That model is not sufficient for AWS CodeArtifact.

CodeArtifact authorization tokens are short-lived. They should be generated inside each GitHub Actions job after assuming an IAM role through OIDC, not stored as repository or organization secrets. This creates two related gaps:

1. Publishing: a private wheel can be published today only with a repository-owned workflow that duplicates part of Rhiza's release pipeline.
2. Consumption: a project depending on a private wheel cannot authenticate before uv lock, uv sync, or another dependency-installing command inside Rhiza's job-level reusable workflows. A caller cannot insert setup steps into a reusable-workflow job, and credentials generated in a separate job do not carry into downstream jobs.

The current custom-feed support therefore handles feeds with durable tokens, but not CodeArtifact's OIDC and short-lived-token model.

Proposed solution

Add AWS CodeArtifact as an optional, first-class package repository provider for both publishing and dependency consumption. Keep it disabled by default and configure it entirely through generic repository or organization variables.

A possible configuration surface is:

Setting	Purpose
PACKAGE_REPOSITORY	none, pypi, codeartifact, or custom
AWS_REGION	AWS region containing the feed
AWS_ROLE_TO_ASSUME_PUBLISH	OIDC role ARN with publish permissions
AWS_ROLE_TO_ASSUME_READ	OIDC role ARN with read-only permissions
CODEARTIFACT_DOMAIN	CodeArtifact domain
CODEARTIFACT_DOMAIN_OWNER	AWS account that owns the domain
CODEARTIFACT_REPOSITORY	CodeArtifact repository
CODEARTIFACT_UV_INDEX	Optional uv index name used by consuming projects

No AWS access key, secret key, or CodeArtifact authorization token should be stored in GitHub.

Publishing

When PACKAGE_REPOSITORY == 'codeartifact', the release workflow should:

1. Retain the existing tag validation, build, SBOM, provenance, and GitHub Release behavior.
2. Reuse the distributions produced by the standard build job rather than rebuilding them in a second workflow.
3. Grant id-token: write only to the publishing job.
4. Assume AWS_ROLE_TO_ASSUME_PUBLISH using aws-actions/configure-aws-credentials.
5. Resolve the upload endpoint at runtime with aws codeartifact get-repository-endpoint --format pypi.
6. Resolve a short-lived authorization token with aws codeartifact get-authorization-token and mask it immediately.
7. Publish the wheel and source distribution with:

uv publish \
  --publish-url "$CODEARTIFACT_ENDPOINT" \
  --username aws \
  --password "$CODEARTIFACT_TOKEN"

8. Fail clearly when required provider settings are absent or when publication fails.
9. Preserve the existing pypi and custom behavior when another provider is selected.

The following standalone workflow shape has been verified end to end. It is included as a
concrete implementation reference; native Rhiza integration should reuse the standard build
job's distributions rather than running uv build a second time.

name: Publish to AWS CodeArtifact

on:
  push:
    tags:
      - "v*"
  workflow_dispatch:
    inputs:
      tag:
        description: "Version tag to publish"
        required: true
        type: string

permissions:
  contents: read

jobs:
  publish:
    if: vars.PACKAGE_REPOSITORY == 'codeartifact'
    runs-on: ubuntu-latest
    permissions:
      contents: read
      id-token: write
    steps:
      - name: Checkout
        uses: actions/checkout@v6.1.0
        with:
          fetch-depth: 0

      - name: Install uv
        uses: astral-sh/setup-uv@v7.6.0

      - name: Build distributions
        run: uv build

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v5
        with:
          role-to-assume: ${{ vars.AWS_ROLE_TO_ASSUME_PUBLISH }}
          aws-region: ${{ vars.AWS_REGION }}

      - name: Resolve CodeArtifact credentials
        id: codeartifact
        shell: bash
        run: |
          endpoint=$(aws codeartifact get-repository-endpoint \
            --domain "${{ vars.CODEARTIFACT_DOMAIN }}" \
            --domain-owner "${{ vars.CODEARTIFACT_DOMAIN_OWNER }}" \
            --repository "${{ vars.CODEARTIFACT_REPOSITORY }}" \
            --format pypi \
            --query repositoryEndpoint \
            --output text)

          token=$(aws codeartifact get-authorization-token \
            --domain "${{ vars.CODEARTIFACT_DOMAIN }}" \
            --domain-owner "${{ vars.CODEARTIFACT_DOMAIN_OWNER }}" \
            --query authorizationToken \
            --output text)

          echo "endpoint=$endpoint" >> "$GITHUB_OUTPUT"
          echo "::add-mask::$token"
          echo "token=$token" >> "$GITHUB_OUTPUT"

      - name: Publish distributions
        shell: bash
        run: |
          uv publish \
            --publish-url "${{ steps.codeartifact.outputs.endpoint }}" \
            --username aws \
            --password "${{ steps.codeartifact.outputs.token }}"

Expected Rhiza release-workflow changes

The integration should update both authoritative GitHub workflow copies:

• .github/workflows/rhiza_release.yml
• bundles/github/.github/workflows/rhiza_release.yml

The existing build job already uploads the wheel and source distribution as the dist
artifact. Add a codeartifact job alongside pypi, with this shape:

• needs: [tag, build, draft-release];
• the existing release environment;
• contents: read and id-token: write permissions;
• the same hardened-runner and checkout conventions as the other release jobs;
• download the existing dist artifact rather than rebuilding;
• validate the selected provider and required CodeArtifact settings;
• assume the publisher role, resolve the endpoint and token, mask the token, and run
  uv publish dist/* against the resolved endpoint;
• expose should_publish and, if useful, package/version outputs for final release notes.

Provider selection should be explicit:

• pypi: retain trusted publishing and reject Private :: Do Not Upload;
• custom: retain the current URL plus durable-token behavior;
• codeartifact: use AWS OIDC and permit packages marked private;
• none: skip package publication while retaining the remaining release outputs.

finalise-release should add codeartifact to needs, include a successful CodeArtifact job
in its completion condition, and report private-feed publication without printing the endpoint
or credentials. The conda job currently depends on public PyPI metadata; it should remain
disabled unless the selected provider supports that lookup.

Update tests/api/test_release_workflow.py to cover the new job, provider branches,
permissions, artifact reuse, step order, and finalization behavior. Existing tests that assert
the live and bundled release workflows remain synchronized should continue to cover both files.
The initial implementation can be GitHub Actions-specific; GitLab behavior should be left
unchanged or explicitly documented as unsupported rather than implying GitHub OIDC works there.

The Private :: Do Not Upload classifier should continue preventing accidental public PyPI publication, but it should not suppress an explicitly selected private CodeArtifact publication. The provider selection and classifier check therefore need to be evaluated together.

The publisher IAM role should be independently configurable and limited to the required token, endpoint, package-version publishing, and package-metadata operations. Its trust policy can restrict assumption to the appropriate repository and release refs.

Consumption

Reusable workflows that install project dependencies should optionally authenticate with CodeArtifact before the first uv operation that can resolve or download project packages.

For each relevant job:

1. Grant id-token: write only when CodeArtifact consumption is enabled.
2. Assume AWS_ROLE_TO_ASSUME_READ with aws-actions/configure-aws-credentials.
3. Request and mask a fresh CodeArtifact authorization token.
4. Expose credentials to uv for the remainder of that job.

The underlying AWS token and uv named-index commands have been verified locally with a private
wheel. The complete job-scoped sequence below has also been verified successfully in GitHub
Actions, including OIDC role assumption, token retrieval, locked synchronization, and package
import:

name: Verify CodeArtifact consumption

on:
  push:
  workflow_dispatch:

permissions:
  contents: read

jobs:
  verify:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      id-token: write
    steps:
      - name: Checkout
        uses: actions/checkout@v6.1.0

      - name: Validate configuration
        shell: bash
        env:
          AWS_REGION: ${{ vars.AWS_REGION }}
          AWS_ROLE_TO_ASSUME_READ: ${{ vars.AWS_ROLE_TO_ASSUME_READ }}
          CODEARTIFACT_DOMAIN: ${{ vars.CODEARTIFACT_DOMAIN }}
          CODEARTIFACT_DOMAIN_OWNER: ${{ vars.CODEARTIFACT_DOMAIN_OWNER }}
          CODEARTIFACT_UV_INDEX: ${{ vars.CODEARTIFACT_UV_INDEX }}
        run: |
          for name in \
            AWS_REGION \
            AWS_ROLE_TO_ASSUME_READ \
            CODEARTIFACT_DOMAIN \
            CODEARTIFACT_DOMAIN_OWNER \
            CODEARTIFACT_UV_INDEX
          do
            if [[ -z "${!name}" ]]; then
              echo "::error::Missing GitHub Actions variable: $name"
              exit 1
            fi
          done

      - name: Install uv
        uses: astral-sh/setup-uv@v10.1.0

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v6.2.4
        with:
          role-to-assume: ${{ vars.AWS_ROLE_TO_ASSUME_READ }}
          aws-region: ${{ vars.AWS_REGION }}

      - name: Authenticate uv with CodeArtifact
        shell: bash
        env:
          AWS_REGION: ${{ vars.AWS_REGION }}
          CODEARTIFACT_DOMAIN: ${{ vars.CODEARTIFACT_DOMAIN }}
          CODEARTIFACT_DOMAIN_OWNER: ${{ vars.CODEARTIFACT_DOMAIN_OWNER }}
          CODEARTIFACT_UV_INDEX: ${{ vars.CODEARTIFACT_UV_INDEX }}
        run: |
          token=$(aws codeartifact get-authorization-token \
            --domain "$CODEARTIFACT_DOMAIN" \
            --domain-owner "$CODEARTIFACT_DOMAIN_OWNER" \
            --region "$AWS_REGION" \
            --query authorizationToken \
            --output text)

          normalized_index=$(printf '%s' "$CODEARTIFACT_UV_INDEX" \
            | tr '[:lower:]' '[:upper:]' \
            | tr -c '[:alnum:]' '_')

          echo "::add-mask::$token"
          echo "UV_INDEX_${normalized_index}_USERNAME=aws" >> "$GITHUB_ENV"
          echo "UV_INDEX_${normalized_index}_PASSWORD=$token" >> "$GITHUB_ENV"

      - name: Sync locked environment
        run: uv sync --all-extras --all-groups --locked

      - name: Verify private package
        run: uv run python -c "import private_package"

For projects using a named explicit index in pyproject.toml, prefer uv's index-specific environment variables:

echo "UV_INDEX_<NORMALIZED_INDEX_NAME>_USERNAME=aws" >> "$GITHUB_ENV"
echo "UV_INDEX_<NORMALIZED_INDEX_NAME>_PASSWORD=$CODEARTIFACT_TOKEN" >> "$GITHUB_ENV"

Here <NORMALIZED_INDEX_NAME> is the configured uv index name uppercased with non-alphanumeric characters converted to underscores. This keeps only private packages on CodeArtifact while public dependencies remain locked to PyPI. It also avoids embedding credentials in a URL.

Authentication must run independently in every job that consumes dependencies; $GITHUB_ENV, AWS session credentials, and uv credentials do not cross job boundaries. This likely includes the reusable CI, documentation, notebook, benchmark, weekly compatibility, devcontainer, and release/SBOM paths wherever they execute uv lock, uv sync, or an equivalent project install.

This should not mean copying the full authentication script into every workflow job. Prefer a
single maintained composite action, for example
jebel-quant/rhiza/.github/actions/codeartifact-auth@<version>, that:

• validates the role, region, domain, owner, and uv index-name inputs;
• invokes aws-actions/configure-aws-credentials;
• requests and masks the CodeArtifact token; and
• writes the normalized uv index credentials to $GITHUB_ENV.

Each consuming workflow job would then contain a small call like:

permissions:
  contents: read
  id-token: write

steps:
  - uses: jebel-quant/rhiza/.github/actions/codeartifact-auth@vX
    with:
      role: ${{ vars.AWS_ROLE_TO_ASSUME_READ }}
      region: ${{ vars.AWS_REGION }}
      domain: ${{ vars.CODEARTIFACT_DOMAIN }}
      domain-owner: ${{ vars.CODEARTIFACT_DOMAIN_OWNER }}
      uv-index-name: ${{ vars.CODEARTIFACT_UV_INDEX }}

  - run: uv sync --all-extras --all-groups --locked

Each dependency-consuming job must still grant id-token: write and invoke that action before
its first project-resolving uv command. A composite action removes duplicated implementation,
but it cannot share credentials between jobs or grant job-level permissions on the caller's
behalf. The workflow tests should enumerate dependency-consuming jobs and assert that each one
contains the authentication action before its install step, preventing new jobs from silently
omitting private-feed setup.

The reader IAM role should be separate from the publisher role and limited to the token, endpoint, repository-read, and package-file operations required for installation.

GitHub OIDC subject configuration

The repository's GitHub OIDC subject format must match the IAM role's trust policy. If the trust
policy uses GitHub's stable organization and repository IDs, enable immutable subjects for each
repository that assumes the role:

gh api --method PUT repos/OWNER/REPOSITORY/actions/oidc/customization/sub \
  -F use_default=true \
  -F use_immutable_subject=true

Verify the resulting prefix before running CI:

gh api repos/OWNER/REPOSITORY/actions/oidc/customization/sub

With immutable subjects disabled, GitHub emits the mutable repo:OWNER/REPOSITORY:... subject.
An IAM policy expecting the immutable repo:OWNER@ORG_ID/REPOSITORY@REPOSITORY_ID:... form then
rejects an otherwise valid token with Not authorized to perform sts:AssumeRoleWithWebIdentity. Rhiza's documentation should show both subject formats and require
the repository setting and IAM condition to agree; it should not prescribe weakening the IAM
trust pattern to work around a mismatched setting.

Pull requests from forks

OIDC access to a private feed must not be exposed to untrusted fork code. The implementation should define an explicit policy for fork pull requests, such as skipping private-package jobs with a clear explanation. Same-repository branches and trusted release refs should continue normally when permitted by the IAM trust policy.

Why the current alternatives are insufficient

• PYPI_TOKEN or UV_EXTRA_INDEX_URL secret: CodeArtifact tokens expire, normally within hours, so a stored value requires continuous secret rotation and will fail unpredictably.
• Generate the token in a prerequisite job: GitHub Actions credentials and environment files do not cross jobs. Passing an authorization token as a job output also expands its exposure and should not be the authentication design.
• Repository-owned workflow copies: These work, but duplicate Rhiza's release and quality workflows and drift as Rhiza evolves.
• Use CodeArtifact as the global uv index: This can rewrite all public dependency sources in uv.lock to the private proxy. Named explicit indexes let only designated private packages use CodeArtifact.
• Static AWS credentials: Long-lived AWS access keys are unnecessary when GitHub OIDC is available and should not be introduced.

Acceptance criteria

• CodeArtifact support is optional and disabled by default.
• Publishing and consumption use GitHub OIDC and short-lived CodeArtifact tokens.
• Publisher and reader roles are configured separately.
• The standard release build artifacts are reused for CodeArtifact publication.
• Selecting CodeArtifact publishes private distributions without enabling public PyPI publication.
• Every reusable workflow job that resolves project dependencies authenticates in that same job.
• Named uv indexes are supported without routing all public dependencies through CodeArtifact.
• Tokens are masked, never persisted as GitHub secrets, never embedded in committed files, and never printed.
• Fork pull requests cannot obtain private-feed credentials.
• Existing PyPI and custom-feed behavior remains unchanged.
• Workflow tests cover provider selection, required settings, permissions, step ordering, fork behavior, and failure paths.
• Documentation includes IAM permissions, repository variables, publishing, consumption, and local-development examples using placeholders.

Additional context

A standalone OIDC workflow has validated the publishing sequence end to end, including endpoint discovery, token generation, and uv publish. A downstream project has also validated local installation through a named explicit uv index. The remaining blocker is integrating those steps into Rhiza's reusable jobs so consumers do not need to fork or duplicate the workflows.

[markrichardson]

Have verified both the publish & consumption steps via this approach. Consumption both locally and on GitHub CI runs

[markrichardson]

@tschm @HarryCampion let me know any thoughts, comment, implementation guidance / context before we kick this off