Pypeline steps for semantic release

So you are using Pypeline and you want to automate your python package release process, then you might want to consider the following steps:

1. CheckCIContext Step:

   • Checks if the current CI context (e.g. Jenkins, Github, etc.) and updates the information in the execution environment to be used by the other steps.

2. CreateReleaseCommit Step:

   • Automates versioning and creates a new release commit and tag based on your commit messages.
   • It is a wrapper for the python-semantic-release tool to be used as Pypeline step.

3. PublishPackage Step:

   • Uses poetry to publish your package to PyPI or another repository.
   • Configures credentials dynamically from environment variables.

4. PublishToOrphanBranch Step:

   • Publishes selected files or folders to an orphan branch (e.g. gh-pages, gen-code).
   • Optionally creates a tag aligned with the semantic-release version tag.

PublishToOrphanBranch

The PublishToOrphanBranch step copies configured files and folders to a separate orphan branch. It runs only when a new release is created (a ReleaseCommit exists in the execution context) and only on CI (not during pull requests or local runs).

Configuration

Option	Type	Default	Description
branch	str	(required)	Name of the orphan branch to publish to
paths	list[str]	[]	Files or folders (relative to repo root) to include
create_tag	bool	true	Whether to create a tag on the orphan branch commit

When create_tag is true, a tag named <branch>-<tag> is created, where <tag> is the semantic-release version tag (e.g. v1.2.0). When false, only the branch is pushed.

Example: GitHub Pages (no tag)

pipeline:
  - step: CheckCIContext
    module: pypeline-semantic-release.steps

  - step: CreateReleaseCommit
    module: pypeline-semantic-release.steps

  - step: PublishToOrphanBranch
    module: pypeline-semantic-release.steps
    config:
      branch: gh-pages
      paths:
        - build/html
      create_tag: false

Example: Generated code (with tag)

pipeline:
  - step: CheckCIContext
    module: pypeline-semantic-release.steps

  - step: CreateReleaseCommit
    module: pypeline-semantic-release.steps

  - step: PublishToOrphanBranch
    module: pypeline-semantic-release.steps
    config:
      branch: gen-code
      paths:
        - generated
        - schema
        - include/api.h
      create_tag: true

How to use it

You need to add this module as a dependency in your pyproject.toml and then use the steps in your pypeline.yaml configuration.

inputs:
  prerelease_token:
    type: string
    description:
      "The prerelease token can be used when multiple users want to create release candidates.
      One can define an own unique token to avoid conflicts: rcN.dev, where N is a digit. For example: rc1.dev"
    required: false
    default: "rc"
  do_prerelease:
    type: boolean
    description: "If set to true, will create a prerelease. This is required to avoid creating prereleases automatically when pushing a branch."
    required: false
    default: false

pipeline:
  - step: CreateVEnv
    module: pypeline.steps.create_venv

  - step: CheckCIContext
    module: pypeline-semantic-release.steps

  - step: CreateReleaseCommit
    module: pypeline-semantic-release.steps

  - step: PublishPackage
    module: pypeline-semantic-release.steps
    config:
      pypi_repository_name: my-repo
      pypi_user_env: PYPI_USER
      pypi_password_env: PYPI_PASSWD

Create releases

The CreateReleaseCommit step will create a release commit based on the semantic-release configuration options in the pyproject.toml file.

[tool.semantic_release.branches.main]
match = "main"
prerelease = false

[tool.semantic_release.branches.feature]
match = "(?!main$)"
prerelease = true

When is a release created?

• When a commit is pushed to the main branch, the step will create a release commit and tag if semantic-release detects a new version shall be created.
• No release is created from a pull request
• No release candidate is created automatically when pushing a branch. This is to avoid creating release candidates when a branch is pushed. See below how to create prereleases.

Create prereleases

Prereleases are only created when the do_prerelease input is set to true. One needs to provide the do_prerelease input when running the pypeline.

Note

To create a prerelease, one needs to push branch and manually trigger a build with the do_prerelease input set to true.

Depending on the CI system, one needs to define input parameters for the users to select when manually triggering the pipeline.

Github Actions

For Github Actions one can define the inputs in the workflow file:

name: CI

on:
  push:
    branches: [develop]
  pull_request:
    branches: [develop]

  workflow_dispatch:
    inputs:
      do_prerelease:
        type: boolean
        description: "If set to true, will create a prerelease.
          This is required to avoid creating prereleases automatically when pushing a branch."
        required: false
        default: false

      prerelease_token:
        description:
          "The prerelease token can be used when multiple users want to create release candidates.
          One can define an own unique token to avoid conflicts: rcN.dev, where N is a digit. For example: rc1.dev"
        required: false
        default: "rc"

Note

The way inputs are defined in Github Actions is similar to how they are defined in Pypeline.

Jenkins

For Jenkins, one can define the inputs in the pipeline script:

properties([
    parameters ([
    booleanParam(
            name: 'do_prerelease',
            defaultValue: false,
            description: '''If set to true, will create a prerelease.
                This is required to avoid creating prereleases automatically when pushing a branch.''',
        ),
    string(
            name: 'prerelease_token',
            defaultValue: '',
            description: '''The prerelease token can be used when multiple users
                want to create release candidates. One can define an own unique token
                to avoid conflicts: rcN.dev, where N is a digit. For example: rc1.dev'''
        ),
    ])
])

Contributing ✨

The project uses Poetry for dependencies management and packaging. Run the bootstrap.ps1 script to install Python and create the virtual environment.

.\bootstrap.ps1

This will also generate a poetry.lock file, you should track this file in version control.

To execute the test suite, call pytest inside Poetry's virtual environment via poetry run:

.venv/Scripts/poetry run pytest

Check out the Poetry documentation for more information on the available commands.

For those using VS Code there are tasks defined for the most common commands:

• bootstrap
• run tests
• run all checks configured for pre-commit

See the .vscode/tasks.json for more details.