-
Docker available locally (required for changelog generation with
git-cliff). -
Create a GitHub Token.
Kubeflow Trainer version format follows Semantic Versioning.
Kubeflow Trainer versions are in the format of vX.Y.Z, where X is the major version, Y is
the minor version, and Z is the patch version.
The patch version contains only bug fixes.
Additionally, Kubeflow Trainer does pre-releases in this format: vX.Y.Z-rc.N where N is a number
of the Nth release candidate (RC) before an upcoming public release named vX.Y.Z.
Kubeflow Trainer releases are tagged with tags like vX.Y.Z, for example v2.0.0.
Release branches are in the format of release-X.Y, where X.Y stands for
the minor release.
vX.Y.Z releases are released from the release-X.Y branch. For example,
v2.0.0 release should be on release-2.0 branch.
If you want to push changes to the release-X.Y release branch, you have to
cherry pick your changes from the master branch and submit a PR.
Kubeflow Trainer uses a directory-based changelog structure under CHANGELOG/:
CHANGELOG/
├── CHANGELOG-1.x.md
├── CHANGELOG-2.0.md
├── CHANGELOG-2.1.md
└── CHANGELOG-2.2.md
Each file contains releases for that minor series. The make release target
prepends new entries automatically using git-cliff.
For the latest minor release, run the following command from the master branch.
For an older minor series patch (for example, v2.2.1 when master is on v2.3.x), checkout
to the corresponding release-X.Y branch and run the following command.
make release VERSION=vX.Y.Z GITHUB_TOKEN=<token>
# or for a release candidate:
make release VERSION=vX.Y.Z-rc.N GITHUB_TOKEN=<token>This will:
- Update
VERSIONtovX.Y.Z. - Update Python API models version to
X.Y.Z - Update Helm Charts version to
X.Y.Z - Generate
CHANGELOG/CHANGELOG-X.Y.mdusinggit-cliff(skipped for RC releases).
After reviewing the changes, create a signed commit and open a PR to the appropriate branch
(e.g. master or release-X.Y):
git add -A && git commit -s -m 'Prepare release vX.Y.Z'When the VERSION change is merged, the
release workflow runs automatically:
- Ensures the
release-X.Ybranch exists and contains the version bump: cherry-picks the merged "Prepare release" commit (VERSION, Helm chart version, regenerated assets, and changelog) onto the branch, or creates the branch frommasterif it doesn't exist yet. - Pins the release-only image references on the release branch, then commits and pushes:
- Image tags (
newTag) in the manifest overlays. CACHE_IMAGEin the data cache runtime.configMapGeneratorversion in the manager overlay.
- Image tags (
- Builds and validates the Python package with
uv. - Creates and pushes the git tag
vX.Y.Z(skipped if it already exists). - Publishes the Python package to PyPI using OIDC trusted publishing.
- Creates a GitHub Release with the generated changelog and the built package artifacts.
Post the release announcement for the new Kubeflow Trainer release in:
- #kubeflow-trainer Slack channel
kubeflow-discussmailing list
When a new minor release branch (release-X.Y) is cut, update the
milestone_applier
configuration in the GoogleCloudPlatform/oss-test-infra repository so Prow keeps
auto-applying the correct milestone to PRs on each branch. See this PR example.
- Bump the
mastermilestone to the next upcoming minor (e.g.v2.2->v2.3). - Add an entry pinning the newly created release branch to its milestone
(e.g.
release-2.2: v2.2).
milestone_applier:
kubeflow/trainer:
master: v2.3
release-2.2: v2.2
release-2.1: v2.1
release-2.0: v2.0