Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 17 additions & 24 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,16 @@
name: Upload Python Package

# Triggers:
# - release: published -> auto-publish when a GH Release is created manually
# (e.g. via the manual fallback in docs/guides/release.md)
# - workflow_dispatch -> manual publish, used for TestPyPI smoke tests or
# recovery (e.g. shipping a tag that release.yml failed
# to publish). Pick the tag/branch in the dispatch UI.
#
# This workflow is NOT called from release.yml -- release.yml inlines its own
# publish steps. See docs/guides/release.md for the rationale (PyPI Trusted
# Publishing attestations are incompatible with reusable workflow chains).

on:
release:
types: [ published ]
Comment thread
punitarani marked this conversation as resolved.
Expand All @@ -13,17 +24,6 @@ on:
options:
- testpypi
- pypi
workflow_call:
inputs:
environment:
description: 'Environment to publish to (pypi or testpypi)'
required: false
default: 'pypi'
type: string
ref:
description: 'Git ref (tag/branch/SHA) to build from. Required when called from release.yml so the build picks up the bump commit, not the caller-event SHA.'
required: false
type: string

permissions:
contents: read
Expand All @@ -33,17 +33,13 @@ permissions:
jobs:
test:
uses: ./.github/workflows/test.yml
with:
ref: ${{ inputs.ref }}

release-build:
needs: [ test ]
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}

- name: Install uv
uses: astral-sh/setup-uv@v7
Expand All @@ -56,14 +52,6 @@ jobs:
- name: Install dependencies
run: uv sync --all-extras

- name: Run tests
run: uv run pytest -v --ignore=tests/search/ -k "not test_search_dates_round_trip"

- name: Check code quality
run: |
uv run ruff format --check .
uv run ruff check .

- name: Build release distributions
run: uv build

Expand All @@ -80,7 +68,12 @@ jobs:
runs-on: ubuntu-latest
needs:
- release-build
if: ${{ github.event_name == 'release' || inputs.environment == 'pypi' }}
# Only auto-publish on release events created by humans (manual fallback).
# release.yml publishes inline and creates its tag/Release via the bot, so
# excluding github-actions[bot] prevents a second, racing publish that would
# 400 with "File already exists". workflow_dispatch publishes only when the
# operator explicitly selects the pypi environment.
if: ${{ (github.event_name == 'workflow_dispatch' && inputs.environment == 'pypi') || (github.event_name == 'release' && github.actor != 'github-actions[bot]') }}
permissions:
id-token: write

Expand Down
57 changes: 47 additions & 10 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -229,21 +229,58 @@ jobs:
cat release_notes.md
} >> "$GITHUB_STEP_SUMMARY"

publish:
# Run the standard test suite against the new tag before publishing.
# Called via workflow_call (not publish), so no OIDC/attestation concerns.
test:
needs: release
if: ${{ needs.release.outputs.did_release == 'true' }}
uses: ./.github/workflows/publish.yml
uses: ./.github/workflows/test.yml
with:
environment: pypi
# Build from the new tag, not the caller-event SHA. workflow_call
# inherits the parent's github.sha (the SHA at release-dispatch time,
# before the bump commit lands), so without this the publish job would
# rebuild the previous version and PyPI would reject it as duplicate.
ref: ${{ needs.release.outputs.tag }}

publish:
needs: [ release, test ]
if: ${{ needs.release.outputs.did_release == 'true' }}
runs-on: ubuntu-latest
permissions:
contents: read
checks: write
pull-requests: write
id-token: write
secrets: inherit
environment:
name: pypi
url: https://pypi.org/p/flights
steps:
# IMPORTANT: build and publish steps live here (inline) rather than
# invoking publish.yml via workflow_call. PyPI's Trusted Publishing
# attestations are incompatible with reusable workflow chains -- the
# OIDC token's job_workflow_ref points at the called workflow while
# the Sigstore cert's Build Config URI points at the top-level
# workflow. PyPI ties both to the same publisher, so chained
# publishes always fail attestation verification.
# See https://docs.pypi.org/trusted-publishers/troubleshooting/#reusable-workflows-on-github
- name: Checkout release tag
uses: actions/checkout@v6
with:
ref: ${{ needs.release.outputs.tag }}

- name: Install uv
uses: astral-sh/setup-uv@v7
with:
version: "latest"

- name: Set up Python
run: uv python install 3.12

- name: Install dependencies
run: uv sync --all-extras

- name: Build release distributions
run: uv build

- name: Check package
run: uv run twine check dist/*

- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
packages-dir: dist/

67 changes: 52 additions & 15 deletions docs/guides/release.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,20 +5,34 @@ on push to `main`; cutting a release is always an explicit, one-click action.

## Overview

The release pipeline is two workflows:
The release pipeline is two independent workflows:

| Workflow | File | Trigger |
| --- | --- | --- |
| **Release** | `.github/workflows/release.yml` | `workflow_dispatch` only |
| **Upload Python Package** | `.github/workflows/publish.yml` | `release: published`, `workflow_dispatch`, `workflow_call` |

`release.yml` bumps the version in `pyproject.toml`, refreshes `uv.lock`,
commits to `main`, creates an annotated tag and GitHub Release, then calls
`publish.yml` to build and upload to PyPI via Trusted Publishing.

`publish.yml` can also be triggered independently — by publishing a GitHub
Release manually, or via `workflow_dispatch` (defaults to TestPyPI for
safe smoke tests).
| **Upload Python Package** | `.github/workflows/publish.yml` | `release: published`, `workflow_dispatch` |

`release.yml` is the end-to-end release pipeline: it bumps the version in
`pyproject.toml`, refreshes `uv.lock`, commits to `main`, creates an annotated
tag and GitHub Release, runs the full test matrix against the new tag, and
builds + uploads to PyPI via Trusted Publishing — all inline, no chained
workflow.

`publish.yml` is a standalone publish workflow used for:
- **Manual recovery** — if `release.yml`'s publish step ever fails, dispatch
`publish.yml` on the failed tag (`environment: pypi`) to ship it.
- **Manual GitHub Release** — if you create a release in the UI from an
existing tag, the `release: published` trigger picks it up and publishes.
- **TestPyPI smoke tests** — `workflow_dispatch` with `environment: testpypi`.

> **Why two workflows instead of one reusable?** PyPI's Trusted Publishing
> attestations are incompatible with reusable workflow chains
> ([pypa/gh-action-pypi-publish#166](https://github.com/pypa/gh-action-pypi-publish/issues/166)).
> When `release.yml` calls `publish.yml` via `workflow_call`, the OIDC token's
> `job_workflow_ref` points at `publish.yml` while the Sigstore cert's
> `Build Config URI` points at `release.yml`; PyPI ties both to the same
> publisher and rejects the attestation. Inlining the publish step into
> `release.yml` sidesteps this entirely.

## Cutting a release

Expand Down Expand Up @@ -65,14 +79,21 @@ modify `pyproject.toml`.

These need to be in place once on the GitHub side:

* **PyPI Trusted Publisher** for the `flights` project, bound to this repo
and the `pypi` environment. `publish.yml` requests an OIDC token via
`id-token: write` and uses `pypa/gh-action-pypi-publish`.
* **PyPI Trusted Publishers** for the `flights` project, both bound to this
repo and the `pypi` environment:
* `release.yml` — used by the automated end-to-end release flow
* `publish.yml` — used by manual recovery, `release: published`, and
TestPyPI dispatch

Configure both at
[pypi.org/manage/project/flights/settings/publishing/](https://pypi.org/manage/project/flights/settings/publishing/).
Set Environment name to `pypi` on both.
* **Branch protection on `main`** must permit pushes from `github-actions[bot]`.
If protection blocks the bot, the release workflow's push will fail; switch
the checkout step's `token:` to a PAT secret instead.
* **Workflow permissions**: `release.yml` requires `contents: write` (already
declared at the workflow level).
declared at the workflow level) and `id-token: write` on the `publish`
job for OIDC.

## Troubleshooting

Expand All @@ -88,10 +109,26 @@ These need to be in place once on the GitHub side:
* **Release notes look wrong on first run** — the workflow walks back to the
last commit whose subject starts with `Bump version`. If you've changed
that convention, edit `release.yml`'s "Determine commit range" step.
* **PyPI returns 400 "Invalid attestations supplied"** with cert URI mismatch
— the publish job is running via a `workflow_call` chain (the original
bug). Make sure the publish job is defined inline in `release.yml`, not
invoked via `uses: ./.github/workflows/publish.yml`. As a one-time recovery
for the failed tag, dispatch `publish.yml` standalone on that tag.

## Manual fallback

If the workflow is broken and a release is urgent, you can still:
If `release.yml`'s publish step fails after the tag and GitHub Release have
already been created (e.g. transient PyPI outage), recover with:

1. Actions → **Upload Python Package** → Run workflow
2. **Branch dropdown**: switch to the failed tag (e.g. `vX.Y.Z`)
3. `environment`: `pypi`
4. Run

This dispatches `publish.yml` standalone and uploads the same tag's artifact.

If the entire `release.yml` workflow is broken and a release is urgent, you
can also:

1. Bump the version in `pyproject.toml` and `uv.lock` on a branch, merge to
`main`.
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "flights"
version = "0.9.0"
version = "0.10.0"
description = "A Python wrapper for Google Flights API"
authors = [
{ name = "Punit Arani", email = "punitsai36@gmail.com" }
Expand Down
2 changes: 1 addition & 1 deletion uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.