diff --git a/.github/README_for_versioning.md b/.github/README_for_versioning.md new file mode 100644 index 000000000..9691d5837 --- /dev/null +++ b/.github/README_for_versioning.md @@ -0,0 +1,297 @@ +# Versioning Freeze — Implementation Guide for chem-dcat-ap + +This folder contains **drop-in artefacts** for implementing the versioning freeze +pipeline in the real [chem-dcat-ap](https://github.com/nfdi-de/chem-dcat-ap) +repository. Everything here is production-ready (using `w3id.org/nfdi-de/…` URLs) +and was developed and validated in the test repos +[test_dcat_ap_plus_versioning_freeze](https://github.com/HendrikBorgelt/test_dcat_ap_plus_versioning_freeze) and +[test_chemDCAT_ap_versioning_freeze](https://github.com/HendrikBorgelt/test_chemDCAT_ap_versioning_freeze). + +**No changes are needed in dcat-ap-plus.** The pipeline is entirely self-contained +within chem-dcat-ap. + +--- + +## The problem being solved + +`chem_dcat_ap.yaml`, `chemical_reaction_ap.yaml`, and `material_entities_ap.yaml` +all import dcat-ap-plus via: + +```yaml +imports: + - dcatapplus:latest/schema/dcat_ap_plus +``` + +`latest` is a **live alias** that mike updates on every dcat-ap-plus release. +This means every previously deployed version of chem-dcat-ap silently switches +to the new dcat-ap-plus the moment it is released — retroactively breaking any +version that was not compatible with the new dcat-ap-plus. + +Two fixes work together: + +1. **Release-time freeze** — at tag push time, the CI pins the import to the + specific dcat-ap-plus version that `latest` resolves to at that moment. + Only the deployed GitHub Pages snapshot carries the pinned import; source + files are never modified. + +2. **Daily upstream check** — a scheduled workflow polls dcat-ap-plus GitHub + Pages once a day, detects any new release, and automatically opens a + compatibility freeze PR so maintainers can verify the new version before + it affects any chem-dcat-ap release. + +--- + +## Folder structure + +``` +for_direct_implementation_at_chemdcat_ap/ + README.md ← This file + workflows/ + deploy-docs.yaml ← Drop-in for .github/workflows/deploy-docs.yaml + handle-upstream-release.yaml ← New workflow (add to .github/workflows/) + scripts/ + freeze_imports.py ← Add to scripts/ in chem-dcat-ap + should_update_latest.py ← Add to scripts/ in chem-dcat-ap +``` + +--- + +## Implementation steps + +### 1. Add the scripts + +Create a `scripts/` directory in the root of chem-dcat-ap (if it does not exist) +and copy both Python files from `scripts/` here into it. + +### 2. Enable Actions to create pull requests (one-time) + +In the repository go to **Settings → Actions → General → Workflow permissions** +and tick **"Allow GitHub Actions to create and approve pull requests"**. + +Without this, the `handle-upstream-release` workflow will fail when it tries +to open the freeze PR. + +### 3. Replace / add the workflows + +- Replace `.github/workflows/deploy-docs.yaml` with `workflows/deploy-docs.yaml` + from this folder. +- Copy `workflows/handle-upstream-release.yaml` into `.github/workflows/` + (this is a new file — it does not replace anything). + +> **Note on action versions:** The production workflow uses stable floating +> tags (`actions/checkout@v4`, `astral-sh/setup-uv@v5`, etc.). Update these +> to whatever your project currently uses if needed. + +### 4. Verify locally (optional but recommended) + +Run the freeze script in dry-run style to confirm it resolves the alias +correctly — then discard the change: + +```bash +# Check which version 'latest' currently resolves to +uv run python scripts/freeze_imports.py \ + --schema-dir src/chem_dcat_ap/schema \ + --prefix dcatapplus \ + --from-alias latest \ + --versions-url https://nfdi-de.github.io/dcat-ap-plus/versions.json + +# Revert — the CI never commits this change, but locally you need to undo it +git checkout src/chem_dcat_ap/schema/ +``` + +### 5. Release as usual + +The release workflow is **unchanged** — just push a version tag: + +```bash +git tag v1.2.0 +git push origin v1.2.0 +``` + +The CI will automatically freeze the import and handle the `latest` alias. + +--- + +## How the pipeline works end to end + +``` +Every day at 08:00 UTC + │ + ▼ +chem-dcat-ap handle-upstream-release.yaml + • Fetch versions.json from nfdi-de.github.io/dcat-ap-plus + • Detect current dcatapplus token in source schemas + • Skip guard A: already up to date? → stop + • Skip guard B: freeze PR already open? → stop + • Create branch freeze/dcatapplus-v1.2.0 + • Commit: dcatapplus:/ → dcatapplus:v1.2.0/ + • Open PR with CI checklist + • Post notice on every open PR + │ + ▼ +Maintainer reviews freeze PR + • CI green → merge; main now deterministically targets v1.2.0 + • CI red → breaking change in v1.2.0; fix schema before merging + │ + ▼ +chem-dcat-ap release: git tag v2.0.0 && git push + │ + ▼ +deploy-docs.yaml (release path) + • freeze_imports.py: source already has dcatapplus:v1.2.0/ → no-op + • just gen-doc → mike deploy v2.0.0 + • deployed schema at /v2.0.0/schema/ has frozen import +``` + +The `handle-upstream-release` workflow can also be triggered manually at any +time via **Actions → Handle upstream dcat-ap-plus release → Run workflow** — +useful for an immediate check or to backfill after a missed schedule run. + +--- + +## Design decisions FAQ + +### Does the schema `id:` need to be versioned? + +**No.** The `id:` field is the stable namespace for concepts defined in the +schema (e.g. `chemdcatap:SubstanceSample`). Versioning it would change the +URI for every concept on every release, breaking RDF compatibility. The +GitHub Pages path already provides versioning context — the deployed file at +`/v1.2.0/schema/chem_dcat_ap.yaml` *is* the versioned artefact. Consumers +who need version-pinned imports reference that path directly. + +### Do tests need versioning? + +**No.** Tests run against the current branch's schema. On the `main` branch +they always use `dcatapplus:latest/`, which is correct for development. At +release time the freeze runs *before* `just gen-doc`, so any schema-level +validation performed during doc generation also exercises the frozen import. + +If you want to additionally run `just test` against the frozen schema in CI +(recommended for critical releases), add a `just test` step *after* the +freeze step in `deploy-docs.yaml`. + +### Does test data need versioning? + +**No.** Test data files (`tests/data/valid/*.yaml`) live on the branch and +are implicitly versioned by git. When a breaking schema change is introduced, +the test data is updated in the same PR. On a maintenance branch (see below) +the test data reflects that version's schema. + +### How do I work on v1.10.4 while v2.0.1 is already live? + +Use a **maintenance branch**: + +```bash +# Create a v1.x maintenance branch from the last v1 tag +git checkout -b v1.x v1.10.3 +``` + +On the `v1.x` branch, **commit the frozen import** to git (unlike the +`main`-branch approach where only CI freezes it). This ensures that +`just test` works correctly locally and in CI for v1.x: + +```bash +# On the v1.x branch, run the freeze and commit the result +uv run python scripts/freeze_imports.py \ + --schema-dir src/chem_dcat_ap/schema \ + --prefix dcatapplus \ + --from-alias latest \ + --versions-url https://nfdi-de.github.io/dcat-ap-plus/versions.json + +git add src/chem_dcat_ap/schema/ +git commit -m "chore: freeze dcatapplus import to for v1.x maintenance" +``` + +Now apply your fix, then release: + +```bash +git tag v1.10.4 +git push origin v1.x v1.10.4 +``` + +The `should_update_latest.py` script will detect that v1.10.4 < v2.0.1 and +**will not overwrite the `latest` alias**. The docs at `/v1.10.4/` will be +deployed cleanly alongside `/v2.0.1/` and `/latest/`. + +--- + +## Summary table + +| Concern | Approach | +|---|---| +| Import freeze scope | `src/chem_dcat_ap/schema/*.yaml` only | +| Release-time freeze | CI working copy only — never committed to `main` | +| Upstream change detection | Daily poll of dcat-ap-plus `versions.json` — no secrets, no changes to dcat-ap-plus | +| Duplicate PR guard | Checks for existing `freeze/dcatapplus-` branch before acting | +| Schema `id:` | Unchanged — unversioned stable namespace | +| `latest` alias promotion | Semver-gated via `should_update_latest.py` | +| Test data | Branch-versioned via git, no explicit versioning needed | +| Maintenance releases | `v1.x` branch + committed frozen import | + +--- + +## Optional tools: compatibility badge and matrix + +These tools surface the health of all deployed versions to **maintainers** +(via a GitHub Issue) and **downstream users** (via a badge in the README and +a compatibility matrix page). + +### Folder structure additions + +``` +for_direct_implementation_at_chemdcat_ap/ + workflows/ + check-schema-compatibility.yaml ← New workflow (add to .github/workflows/) + scripts/ + check_compatibility.py ← New script (add to scripts/) +``` + +### What the workflow does + +Runs every **Monday at 06:00 UTC** (plus `workflow_dispatch` for manual runs): + +1. Checks every deployed version's frozen `dcatapplus:vX.Y.Z/` import against + dcat-ap-plus GitHub Pages (HTTP HEAD request — no auth needed). +2. Generates `badge.json` and `compatibility.html` and pushes them to the + **gh-pages root** as stable, version-independent URLs: + - `https://nfdi-de.github.io/chem-dcat-ap/badge.json` + - `https://nfdi-de.github.io/chem-dcat-ap/compatibility.html` +3. **Issue on breakage** — if `latest` has newly become stale, opens a + `schema-dep-stale` issue to notify maintainers. The label is created + automatically if it does not exist. +4. **Auto-closes** the issue once `latest` is valid again. + +`dev` is excluded from all checks and counts — it has a floating import by design. + +### Badge color scale + +| Badge | Condition | +|---|---| +| 🟢 `all valid` | Every released version's upstream resolves | +| 🟡🟢 `N version(s) stale` | Some older versions stale, `latest` valid, < 50% total | +| 🟡 `N versions stale` | ≥ 50% of released versions stale, `latest` valid | +| 🔴 `latest stale` | The `latest`-aliased version has a broken import | + +### Implementation steps + +#### 1. Copy the files + +- `scripts/check_compatibility.py` → `scripts/check_compatibility.py` +- `workflows/check-schema-compatibility.yaml` → `.github/workflows/check-schema-compatibility.yaml` + +#### 2. Add the badge to the README + +```markdown +[![schema deps](https://img.shields.io/endpoint?url=https://nfdi-de.github.io/chem-dcat-ap/badge.json&style=flat-square)](https://nfdi-de.github.io/chem-dcat-ap/compatibility.html) +``` + +#### 3. Run once manually to initialise + +After merging, trigger the workflow manually via +**Actions → Check schema compatibility → Run workflow** +to generate the initial `badge.json` and `compatibility.html` before the +first scheduled run. + +> The `schema-dep-stale` label and the gh-pages files are all created +> automatically on the first run — no manual setup beyond copying the files. diff --git a/.github/workflows/check-schema-compatibility.yaml b/.github/workflows/check-schema-compatibility.yaml new file mode 100644 index 000000000..886ec62a6 --- /dev/null +++ b/.github/workflows/check-schema-compatibility.yaml @@ -0,0 +1,191 @@ +--- +name: Check schema compatibility + +# Runs weekly to verify that every deployed version of chem-dcat-ap still +# has a resolvable dcat-ap-plus dependency. +# +# What this workflow does +# ----------------------- +# 1. Runs scripts/check_compatibility.py (pure stdlib — no pip install needed). +# 2. Pushes badge.json and compatibility.html directly to the gh-pages root +# so they are available at stable, version-independent URLs. +# 3. If the 'latest' version has newly become stale, opens a GitHub Issue +# labelled 'schema-dep-stale' to notify maintainers. +# 4. If the 'latest' version is now valid and a stale issue is open, +# closes it automatically with a resolution comment. +# +# Stable URLs produced: +# badge.json → https://nfdi-de.github.io/chem-dcat-ap/badge.json +# compatibility.html → https://nfdi-de.github.io/chem-dcat-ap/compatibility.html +# +# No secrets required beyond the default GITHUB_TOKEN. + +on: # yamllint disable-line rule:truthy + schedule: + - cron: '0 6 * * 1' # Weekly on Mondays at 06:00 UTC + workflow_dispatch: # manual trigger + +permissions: {} + +jobs: + check: + runs-on: ubuntu-latest + permissions: + contents: write # push badge.json + compatibility.html to gh-pages + issues: write # open / close the schema-dep-stale issue + + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + steps: + - name: Checkout + uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - name: Configure git for the bot + run: | + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + + # ------------------------------------------------------------------ + # Run the compatibility check. + # Pure stdlib — no uv / pip install needed. + # '|| true' ensures the workflow never fails; state changes are + # communicated through issues, not workflow status. + # ------------------------------------------------------------------ + - name: Run compatibility check + id: compat + run: | + python3 scripts/check_compatibility.py \ + --chemdcatap-url https://nfdi-de.github.io/chem-dcat-ap \ + --dcatapplus-url https://nfdi-de.github.io/dcat-ap-plus \ + --output-dir /tmp/compat_output || true + cat /tmp/compat_output/status.env >> "$GITHUB_OUTPUT" + + # ------------------------------------------------------------------ + # Push badge.json and compatibility.html to the gh-pages root. + # Uses a git worktree so the main checkout is untouched. + # [skip ci] prevents a recursive gh-pages build trigger. + # ------------------------------------------------------------------ + - name: Publish badge and compatibility matrix to gh-pages + run: | + git fetch origin gh-pages + git worktree add /tmp/gh-pages gh-pages + + cp /tmp/compat_output/badge.json /tmp/gh-pages/badge.json + cp /tmp/compat_output/compatibility.html /tmp/gh-pages/compatibility.html + + cd /tmp/gh-pages + git add badge.json compatibility.html + if git diff --staged --quiet; then + echo "No changes to badge or matrix — nothing to commit." + else + git commit -m "chore: update schema compatibility status [skip ci]" + git push origin gh-pages + echo "Pushed updated badge and compatibility matrix." + fi + cd - + git worktree remove /tmp/gh-pages + + # ------------------------------------------------------------------ + # Commit compatibility.md to main so the MkDocs page stays current. + # Uses the main checkout (not the gh-pages worktree). + # No [skip ci] — we want deploy-docs to run so the dev docs on + # gh-pages are rebuilt with the updated compatibility page. + # ------------------------------------------------------------------ + - name: Commit compatibility.md to main branch + run: | + cp /tmp/compat_output/compatibility.md docs/compatibility.md + git add docs/compatibility.md + if git diff --staged --quiet; then + echo "No changes to compatibility.md — nothing to commit." + else + git commit -m "chore: update schema compatibility matrix" + git push origin main + echo "Committed updated compatibility.md to main." + fi + + # ------------------------------------------------------------------ + # Create the label if it does not exist yet (idempotent). + # ------------------------------------------------------------------ + - name: Ensure schema-dep-stale label exists + run: | + gh label create "schema-dep-stale" \ + --color "e4e669" \ + --description "Latest released version depends on an unavailable upstream schema" \ + 2>/dev/null || true + + # ------------------------------------------------------------------ + # Open a new issue only on the first run where latest is stale. + # ------------------------------------------------------------------ + - name: Open issue if latest has newly become stale + if: steps.compat.outputs.latest_stale == 'true' + run: | + EXISTING=$(gh issue list \ + --label "schema-dep-stale" \ + --state open \ + --json number \ + --jq '.[0].number' 2>/dev/null || echo "") + + if [ -z "$EXISTING" ]; then + gh issue create \ + --title "⚠️ Schema dependency stale: latest chem-dcat-ap depends on an unavailable dcat-ap-plus version" \ + --label "schema-dep-stale" \ + --body "$(cat </dev/null || echo "") + + if [ -n "$EXISTING" ]; then + gh issue close "$EXISTING" \ + --comment "$(cat < versioned w3id.org URL (/chemistry/{version}/) + # 2. dcatapplus: prefix -> pinned / re-pinned upstream version + # 3. Bare local imports -> chemdcatap:schema/ CURIE form + # + # NOTE — id: fields are NOT versioned. They are stable RDF namespace + # identifiers (canonical w3id.org URIs) — versioning them would break + # RDF compatibility for all consumers of the schema's named classes. + # + # NOTE — chemdcatap base URL: + # The source schemas use chemdcatap: https://w3id.org/nfdi-de/dcat-ap-plus/chemistry/ + # The freeze appends the version token to produce: + # chemdcatap: https://w3id.org/nfdi-de/dcat-ap-plus/chemistry/{version}/ + # The --chemdcatap-base flag must therefore be the w3id.org path WITH + # the /chemistry suffix, matching the source prefix declaration. + # ----------------------------------------------------------------------- + - name: Freeze schemas for release + if: github.ref_type == 'tag' && github.event_name == 'push' + run: | + echo "=== Schema prefix/import lines BEFORE freeze ===" + for f in src/chem_dcat_ap/schema/*.yaml; do + echo "--- $(basename $f) ---" + grep -E '^\s+(chemdcatap|dcatapplus):|^id:|^imports:|\s+- ' "$f" | head -20 + done + + uv run python scripts/freeze_imports.py \ + --schema-dir src/chem_dcat_ap/schema \ + --dcatapplus-base https://w3id.org/nfdi-de/dcat-ap-plus \ + --versions-url https://nfdi-de.github.io/dcat-ap-plus/versions.json \ + --chemdcatap-base https://w3id.org/nfdi-de/dcat-ap-plus/chemistry \ + --chemdcatap-version ${{ github.ref_name }} \ + --convert-bare-imports \ + --sub-module-base https://w3id.org/nfdi-de/dcat-ap-plus + + echo "=== Schema prefix/import lines AFTER freeze ===" + for f in src/chem_dcat_ap/schema/*.yaml; do + echo "--- $(basename $f) ---" + grep -E '^\s+(chemdcatap|dcatapplus):|^id:|^imports:|\s+- ' "$f" | head -20 + done + + # gen-doc (_add-artifacts) already copied the raw source YAMLs into + # docs/schema/ before this step ran. Overwrite those stale copies + # with the freshly frozen versions so mike deploy publishes the right + # files to gh-pages. + echo "=== Updating docs/schema/ with frozen versions ===" + cp src/chem_dcat_ap/schema/*.yaml docs/schema/ + echo "docs/schema/ now contains:" + grep -E '^\s+(chemdcatap|dcatapplus):|^id:|^imports:|\s+- ' docs/schema/*.yaml | head -60 + # The if-conditions below make sure to select the right step, depending # on the job trigger. Only one of the steps below will run at a time. # The others will be skipped. @@ -76,14 +139,253 @@ jobs: # yamllint disable rule:line-length - name: Build & deploy "dev" docs for a new commit to main - if: (github.event_name == 'push' && github.ref_type != 'tag') || github.event_name == 'workflow_dispatch' run: | export SHORT_SHA=$(echo "${GITHUB_SHA}" | cut -c1-7) uv run mike deploy --push --update-aliases --title "dev (${SHORT_SHA})" dev + # ----------------------------------------------------------------------- + # RELEASE DEPLOYMENT + # + # Deploys the FROZEN working copy (schemas were frozen in the step above). + # Downstream consumers and the post-deploy-validate job will therefore + # see consistently versioned schemas as soon as gh-pages is live. + # + # Only promotes 'latest' when this version is semver >= the version + # currently carrying the 'latest' alias, to prevent a maintenance patch + # on an old branch from overwriting a newer release's 'latest'. + # ----------------------------------------------------------------------- - name: Build & deploy docs for a new version tag if: github.ref_type == 'tag' && github.event_name == 'push' run: | - uv run mike deploy --push --update-aliases ${{ github.ref_name }} latest - uv run mike set-default latest --push + # Deploy under the version path (e.g. /v0.2.0/) + uv run mike deploy --push --update-aliases ${{ github.ref_name }} + + # Check whether to also promote the 'latest' alias + SHOULD_UPDATE=$(uv run python scripts/should_update_latest.py \ + --new-version ${{ github.ref_name }} \ + --versions-url https://nfdi-de.github.io/chem-dcat-ap/versions.json \ + 2>/dev/null || echo "true") + + echo "Promote 'latest' alias: ${SHOULD_UPDATE}" + + if [ "${SHOULD_UPDATE}" = "true" ]; then + uv run mike deploy --push --update-aliases ${{ github.ref_name }} latest + uv run mike set-default latest --push + echo "Updated 'latest' -> ${{ github.ref_name }}" + else + echo "Keeping 'latest' as-is (current latest is a newer version)." + fi + + # ------------------------------------------------------------------------- + # POST-DEPLOY VALIDATION (release tags only) + # + # Runs AFTER build-docs, which already deployed the FROZEN schemas to + # gh-pages. This job therefore only needs to: + # 1. Wait for GitHub Pages to serve the new files. + # 2. Re-apply the freeze locally (fresh checkout of main — idempotent, + # same freeze as Phase 1). + # 3. Validate with gen-yaml: sub-schema imports are fetched from gh-pages, + # which now carries consistently versioned prefixes → no mismatch. + # 4. Push frozen schemas to reference branch schema-release/{tag} + # (always, even on failure — for maintainer inspection). + # 5. Open a GitHub Issue on validation failure. + # ------------------------------------------------------------------------- + post-deploy-validate: + runs-on: ubuntu-latest + needs: [build-docs] + if: github.ref_type == 'tag' && github.event_name == 'push' + permissions: + contents: write + issues: write + + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + steps: + - name: Checkout + uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - name: Configure git for the bot + run: | + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + + - name: Install uv + uses: astral-sh/setup-uv@v5 + with: + python-version: "3.13" + enable-cache: true + cache-dependency-glob: "uv.lock" + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: "3.13" + + - name: Install dependencies + run: uv sync --dev --no-progress + + # ----------------------------------------------------------------------- + # Wait for GitHub Pages to serve the frozen schemas before validating. + # GitHub can take several minutes to rebuild gh-pages after a push. + # Polling is safer than a fixed sleep and exits early on fast deploys. + # + # We probe chem_dcat_ap.yaml (the top-level schema) — a reliable signal + # that the full versioned directory is live. + # Polls every 30 s, times out after 10 minutes (20 attempts). + # ----------------------------------------------------------------------- + - name: Wait for gh-pages deploy to be live + env: + VERSION: ${{ github.ref_name }} + run: | + PROBE_URL="https://nfdi-de.github.io/chem-dcat-ap/${VERSION}/schema/chem_dcat_ap.yaml" + echo "Polling for: ${PROBE_URL}" + for i in $(seq 1 20); do + if curl --silent --fail --max-time 10 "${PROBE_URL}" > /dev/null 2>&1; then + echo "gh-pages is live (attempt ${i})." + exit 0 + fi + echo " Not yet available — waiting 30 s (attempt ${i}/20)..." + sleep 30 + done + echo "ERROR: gh-pages did not become available within 10 minutes." >&2 + exit 1 + + # ----------------------------------------------------------------------- + # Re-apply the full freeze to this fresh checkout of main. + # This re-applies the same freeze as Phase 1 (build-docs) and is + # idempotent when run against the development-form source files on main. + # The locally-frozen files are used to push the reference branch and to + # run gen-yaml validation (which fetches sub-schemas from gh-pages). + # + # id: fields are NOT versioned — they are stable RDF namespace identifiers. + # ----------------------------------------------------------------------- + - name: Apply full freeze + run: | + uv run python scripts/freeze_imports.py \ + --schema-dir src/chem_dcat_ap/schema \ + --dcatapplus-base https://w3id.org/nfdi-de/dcat-ap-plus \ + --versions-url https://nfdi-de.github.io/dcat-ap-plus/versions.json \ + --chemdcatap-base https://w3id.org/nfdi-de/dcat-ap-plus/chemistry \ + --chemdcatap-version ${{ github.ref_name }} \ + --convert-bare-imports \ + --sub-module-base https://w3id.org/nfdi-de/dcat-ap-plus + + # ----------------------------------------------------------------------- + # Validate the frozen schemas. + # gen-yaml resolves chemdcatap: imports by fetching from gh-pages. + # Because build-docs already deployed the frozen versions, those files + # carry the same versioned chemdcatap: prefix — no mismatch. + # ----------------------------------------------------------------------- + - name: Validate fully-frozen schemas + id: validate + run: | + echo "Validating fully-frozen schema files..." + for f in src/chem_dcat_ap/schema/*.yaml; do + echo " checking: $f" + uv run gen-yaml "$f" > /dev/null + done + echo "All frozen schemas are valid." + + # ----------------------------------------------------------------------- + # Push the frozen schemas to a reference branch for inspection. + # Runs always (even on validation failure) so maintainers can inspect + # the frozen state and debug any issues. + # ----------------------------------------------------------------------- + - name: Push frozen schemas to reference branch + if: always() + run: | + BRANCH="schema-release/${{ github.ref_name }}" + git checkout -b "${BRANCH}" + git add src/chem_dcat_ap/schema/ + git commit -m "chore: frozen schemas for ${{ github.ref_name }} + + Full-freeze snapshot generated by the post-deploy-validate workflow. + Bare imports converted to CURIEs, chemdcatap: and dcatapplus: prefixes + versioned. id: fields unchanged (stable RDF namespace identifiers). + + Source files on main are unchanged (development form retained)." + git push origin "${BRANCH}" + + # ----------------------------------------------------------------------- + # Record the freeze-validation outcome in freeze-status.json on gh-pages. + # Runs always so both success and failure are captured. + # The check-schema-compatibility workflow reads this file to populate the + # "Freeze validated" column in the compatibility matrix. + # ----------------------------------------------------------------------- + - name: Publish freeze-status to gh-pages + if: always() + env: + VERSION: ${{ github.ref_name }} + VALIDATED: ${{ steps.validate.outcome == 'success' }} + run: | + git fetch origin gh-pages + git worktree add /tmp/ghp gh-pages + + python3 - <<'PYEOF' + import json, os, pathlib + from datetime import datetime, timezone + f = pathlib.Path("/tmp/ghp/freeze-status.json") + data = json.loads(f.read_text()) if f.exists() else {} + data[os.environ["VERSION"]] = { + "validated": os.environ["VALIDATED"] == "true", + "timestamp": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"), + } + f.write_text(json.dumps(data, indent=2) + "\n") + PYEOF + + cd /tmp/ghp + git add freeze-status.json + if git diff --staged --quiet; then + echo "freeze-status.json unchanged — nothing to commit." + else + git commit -m "chore: update freeze status for ${VERSION} [skip ci]" + git push origin gh-pages + echo "Pushed updated freeze-status.json." + fi + cd - + git worktree remove /tmp/ghp + + # ----------------------------------------------------------------------- + # Notify maintainers when validation fails. + # ----------------------------------------------------------------------- + - name: Open issue on validation failure + if: failure() + run: | + gh issue create \ + --title "Schema post-deploy validation failed for ${{ github.ref_name }}" \ + --label "bug" \ + --body "$(cat <<'EOF' + ## Post-deploy schema validation failed ❌ + + The fully-frozen schemas for **${{ github.ref_name }}** failed \`gen-yaml\` + validation after deploy. + + **What to check:** + - Do the CURIE imports in the frozen schemas resolve correctly? + - Are there breaking changes in the upstream dcat-ap-plus release? + + **Reference branch for inspection:** + \`schema-release/${{ github.ref_name }}\` — contains the frozen schema + state at the time of failure. + + The gh-pages schema files were deployed by Phase 1 (build-docs) and + remain live. The reference branch contains the locally-frozen state + for debugging. + + --- + *This issue was opened automatically by the \`post-deploy-validate\` job.* + EOF + )" + + - name: Job summary + if: success() + run: | + echo "## Post-deploy validation passed ✅" >> "$GITHUB_STEP_SUMMARY" + echo "" >> "$GITHUB_STEP_SUMMARY" + echo "Fully-frozen schemas for **${{ github.ref_name }}** are live on gh-pages." >> "$GITHUB_STEP_SUMMARY" + echo "" >> "$GITHUB_STEP_SUMMARY" + echo "Reference branch: \`schema-release/${{ github.ref_name }}\`" >> "$GITHUB_STEP_SUMMARY" diff --git a/.github/workflows/handle-upstream-release.yaml b/.github/workflows/handle-upstream-release.yaml new file mode 100644 index 000000000..31589be0f --- /dev/null +++ b/.github/workflows/handle-upstream-release.yaml @@ -0,0 +1,323 @@ +--- +name: Handle upstream dcat-ap-plus release + +# Runs daily at 08:00 UTC to check whether dcat-ap-plus has released a new +# version since the last time the source schemas were frozen. +# +# No secrets required. No changes needed in the upstream dcat-ap-plus repo. +# The workflow is fully self-contained: it fetches the current 'latest' version +# from the dcat-ap-plus GitHub Pages and decides autonomously whether to act. +# +# What this workflow does +# ----------------------- +# 1. Fetches the current 'latest' version from dcat-ap-plus GitHub Pages. +# 2. Detects the current dcatapplus import token in the source schemas. +# 3. Skip guard A: if source is already pinned to the upstream latest, stop. +# 4. Skip guard B: if an open PR for this version already exists, stop silently. +# 5. Check whether the freeze branch already exists (e.g. from a previous +# failed run) — branch creation is skipped if it does, but the PR step +# still runs so nothing is lost. +# 6. Create branch freeze/dcatapplus- (if it does not exist yet), +# commit the updated import, open a PR with a CI checklist, and post a +# notice on every open non-freeze PR so contributors are aware of the +# upstream change. +# +# One-time repository setup required (Settings → Actions → General): +# "Allow GitHub Actions to create and approve pull requests" must be enabled. +# +# The workflow_dispatch trigger allows manual runs at any time — useful for +# testing or to trigger an immediate check without waiting for the schedule. + +on: # yamllint disable-line rule:truthy + schedule: + - cron: '0 8 * * *' # daily at 08:00 UTC + workflow_dispatch: # manual trigger — no inputs needed + +permissions: {} + +jobs: + check-and-freeze: + runs-on: ubuntu-latest + permissions: + contents: write # push the freeze branch + pull-requests: write # open the PR and post comments + + # GH_TOKEN at job level so gh CLI is authenticated in all steps, + # including the early skip guards (before tooling is installed). + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + steps: + - name: Checkout + uses: actions/checkout@v4 + with: + fetch-depth: 0 + + # ------------------------------------------------------------------ + # Fetch the current 'latest' version from dcat-ap-plus GitHub Pages + # ------------------------------------------------------------------ + - name: Fetch upstream latest version + id: upstream + run: | + VERSION=$(python3 - <<'EOF' + import json, sys, urllib.request + url = "https://nfdi-de.github.io/dcat-ap-plus/versions.json" + try: + with urllib.request.urlopen(url, timeout=15) as r: + versions = json.load(r) + for entry in versions: + if "latest" in entry.get("aliases", []): + print(entry["version"]) + sys.exit(0) + print("ERROR: no 'latest' alias found in versions.json", file=sys.stderr) + sys.exit(1) + except Exception as exc: + print(f"ERROR: {exc}", file=sys.stderr) + sys.exit(1) + EOF + ) + echo "version=${VERSION}" >> "$GITHUB_OUTPUT" + echo "Upstream latest: ${VERSION}" + + # ------------------------------------------------------------------ + # Detect what version the source schemas currently pin dcatapplus to. + # + # The canonical form on main is: version in the prefix value + # dcatapplus: https://.../v0.5.0/ + # and the import path has no version token: + # - dcatapplus:schema/dcat_ap_plus + # + # If dcatapplus is still at the base URL (unversioned — only expected + # before the first handle-upstream-release PR has been merged), the + # detect step returns "unversioned", which never equals any upstream + # version string and therefore always triggers the PR creation. + # ------------------------------------------------------------------ + - name: Detect current dcatapplus import token + id: detect + run: | + CURRENT=$(python3 - <<'EOF' + import re, pathlib, sys + for f in sorted(pathlib.Path("src/chem_dcat_ap/schema").glob("*.yaml")): + m = re.search(r'dcatapplus:\s+\S+/(v[\d.]+)/', f.read_text()) + if m: + print(m.group(1)) + sys.exit(0) + # Fallback: dcatapplus is at base URL, not yet pinned to any version + print("unversioned") + EOF + ) + echo "current_token=${CURRENT}" >> "$GITHUB_OUTPUT" + echo "Current source token: ${CURRENT}" + + # ------------------------------------------------------------------ + # Skip guard A: source already pinned to the upstream latest + # ------------------------------------------------------------------ + - name: Check if source is already up to date + id: check_uptodate + env: + VERSION: ${{ steps.upstream.outputs.version }} + CURRENT_TOKEN: ${{ steps.detect.outputs.current_token }} + run: | + if [ "${CURRENT_TOKEN}" = "${VERSION}" ]; then + echo "skip=true" >> "$GITHUB_OUTPUT" + echo "Source already pinned to ${VERSION} — nothing to do." + else + echo "skip=false" >> "$GITHUB_OUTPUT" + echo "Update needed: ${CURRENT_TOKEN} → ${VERSION}" + fi + + # ------------------------------------------------------------------ + # Skip guard B: an open freeze PR for this version already exists. + # Uses gh CLI (pre-installed on ubuntu-latest, authenticated via + # GH_TOKEN set at job level). + # Note: we check for an open PR, not just for the branch — a branch + # can exist without a PR (e.g. from a previous run that failed after + # the branch push). In that case we still want to create the PR. + # ------------------------------------------------------------------ + - name: Check if open freeze PR already exists + id: check_pr + if: steps.check_uptodate.outputs.skip == 'false' + env: + VERSION: ${{ steps.upstream.outputs.version }} + run: | + BRANCH="freeze/dcatapplus-${VERSION}" + EXISTING=$(gh pr list \ + --head "${BRANCH}" \ + --state open \ + --json number \ + --jq '.[0].number' 2>/dev/null || echo "") + if [ -n "$EXISTING" ]; then + echo "skip=true" >> "$GITHUB_OUTPUT" + echo "Open freeze PR #${EXISTING} already exists for '${BRANCH}' — skipping." + else + echo "skip=false" >> "$GITHUB_OUTPUT" + echo "No open freeze PR found — will proceed." + fi + + # ------------------------------------------------------------------ + # Check whether the freeze branch already exists (e.g. from a + # previous run that failed after the branch push but before the PR). + # This is separate from the skip guard — if the branch exists but no + # PR does, we still proceed to create the PR. + # ------------------------------------------------------------------ + - name: Check if freeze branch already exists + id: check_branch + if: > + steps.check_uptodate.outputs.skip == 'false' && + steps.check_pr.outputs.skip == 'false' + env: + VERSION: ${{ steps.upstream.outputs.version }} + run: | + BRANCH="freeze/dcatapplus-${VERSION}" + if git ls-remote --exit-code --heads origin "${BRANCH}" > /dev/null 2>&1; then + echo "exists=true" >> "$GITHUB_OUTPUT" + echo "Branch '${BRANCH}' already exists — will skip creation, proceed to PR." + else + echo "exists=false" >> "$GITHUB_OUTPUT" + echo "Branch does not exist — will create it." + fi + + # ------------------------------------------------------------------ + # Install tooling — only when we are actually going to create a PR + # ------------------------------------------------------------------ + - name: Install uv + if: > + steps.check_uptodate.outputs.skip == 'false' && + steps.check_pr.outputs.skip == 'false' + uses: astral-sh/setup-uv@v5 + with: + python-version: "3.13" + enable-cache: true + cache-dependency-glob: "uv.lock" + + - name: Set up Python + if: > + steps.check_uptodate.outputs.skip == 'false' && + steps.check_pr.outputs.skip == 'false' + uses: actions/setup-python@v5 + with: + python-version: "3.13" + + - name: Configure git for the bot + if: > + steps.check_uptodate.outputs.skip == 'false' && + steps.check_pr.outputs.skip == 'false' + run: | + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + + # ------------------------------------------------------------------ + # Create the freeze branch and commit the updated imports. + # Skipped if the branch already exists (recovery path). + # ------------------------------------------------------------------ + - name: Create freeze branch and commit + if: > + steps.check_uptodate.outputs.skip == 'false' && + steps.check_pr.outputs.skip == 'false' && + steps.check_branch.outputs.exists == 'false' + id: branch + env: + VERSION: ${{ steps.upstream.outputs.version }} + CURRENT_TOKEN: ${{ steps.detect.outputs.current_token }} + run: | + BRANCH="freeze/dcatapplus-${VERSION}" + git checkout -b "${BRANCH}" + + uv run python scripts/freeze_imports.py \ + --schema-dir src/chem_dcat_ap/schema \ + --dcatapplus-base https://w3id.org/nfdi-de/dcat-ap-plus \ + --dcatapplus-version "${VERSION}" + + git add src/chem_dcat_ap/schema/ + git commit -m "chore: freeze dcatapplus prefix to ${VERSION} + + Automatically created by the daily upstream check workflow. + + Pins the dcatapplus prefix value from the unversioned base URL + to 'https://w3id.org/nfdi-de/dcat-ap-plus/${VERSION}/', making + all future releases of chem-dcat-ap deterministic against this + upstream version." + + git push origin "${BRANCH}" + + # ------------------------------------------------------------------ + # Open the PR — always runs if both skip guards passed, regardless + # of whether the branch was just created or already existed. + # ------------------------------------------------------------------ + - name: Create pull request + if: > + steps.check_uptodate.outputs.skip == 'false' && + steps.check_pr.outputs.skip == 'false' + id: create-pr + env: + VERSION: ${{ steps.upstream.outputs.version }} + CURRENT_TOKEN: ${{ steps.detect.outputs.current_token }} + run: | + BRANCH="freeze/dcatapplus-${VERSION}" + PR_URL=$(gh pr create \ + --title "chore: freeze dcatapplus prefix to ${VERSION}" \ + --base main \ + --head "${BRANCH}" \ + --body "$(cat <> "$GITHUB_OUTPUT" + echo "Created freeze PR: ${PR_URL}" + + # ------------------------------------------------------------------ + # Post a notice on all open non-freeze PRs + # ------------------------------------------------------------------ + - name: Post notice on open PRs + if: > + steps.check_uptodate.outputs.skip == 'false' && + steps.check_pr.outputs.skip == 'false' + env: + VERSION: ${{ steps.upstream.outputs.version }} + FREEZE_PR_URL: ${{ steps.create-pr.outputs.pr_url }} + run: | + gh pr list --state open --json number,headRefName \ + --jq '.[] | select(.headRefName | startswith("freeze/") | not) | .number' \ + | while read -r PR_NUM; do + gh pr comment "${PR_NUM}" --body "$(cat < list | dict: + req = urllib.request.Request( + url, headers={"User-Agent": "chem-dcat-ap-compat-check/1.0"} + ) + with urllib.request.urlopen(req, timeout=15) as r: + return json.load(r) + + +def fetch_text(url: str) -> str | None: + try: + req = urllib.request.Request( + url, headers={"User-Agent": "chem-dcat-ap-compat-check/1.0"} + ) + with urllib.request.urlopen(req, timeout=15) as r: + return r.read().decode("utf-8") + except Exception: + return None + + +def url_accessible(url: str) -> bool: + """Return True if URL responds with HTTP 2xx (HEAD request).""" + try: + req = urllib.request.Request( + url, + method="HEAD", + headers={"User-Agent": "chem-dcat-ap-compat-check/1.0"}, + ) + with urllib.request.urlopen(req, timeout=15) as r: + return r.status < 300 + except Exception: + return False + + +# --------------------------------------------------------------------------- +# Freeze-status lookup +# --------------------------------------------------------------------------- + +def fetch_freeze_status(chemdcatap_base: str) -> dict: + """Fetch freeze-status.json from gh-pages. Returns {} on any error. + The file records whether post-deploy-validate succeeded for each released + version. Keys are version strings (e.g. 'v0.3.0'). + """ + try: + req = urllib.request.Request( + f"{chemdcatap_base}/freeze-status.json", + headers={"User-Agent": "chem-dcat-ap-compat-check/1.0"}, + ) + with urllib.request.urlopen(req, timeout=15) as r: + return json.load(r) + except Exception: + return {} + + +# --------------------------------------------------------------------------- +# Schema parsing +# --------------------------------------------------------------------------- + +def extract_dcatapplus_version(schema_text: str) -> str | None: + """Extract the pinned dcatapplus version from either: + - New format: dcatapplus: https://.../vX.Y.Z/ (version in prefix value) + - Old format (legacy): dcatapplus:vX.Y.Z/schema/ (version in import path) + Returns the version string (e.g. 'v0.3.0') or None if not pinned. + """ + # New format: version in prefix value + m = re.search(r'dcatapplus:\s+https?://\S+/(v[\d.]+)/', schema_text) + if m: + return m.group(1) + # Old format (legacy): version in import path + m = re.search(r'dcatapplus:(v[\d.]+)/schema/', schema_text) + if m: + return m.group(1) + return None + + +# Keep old name as alias for backwards compatibility with any callers +extract_dcatapplus_token = extract_dcatapplus_version + + +# --------------------------------------------------------------------------- +# Badge +# --------------------------------------------------------------------------- + +def compute_badge(stale_count: int, total: int, latest_stale: bool) -> dict: + """ + Color scale: + red → latest version stale (critical) + yellow → ≥50% of released versions stale, latest valid + yellowgreen → some stale but <50%, latest valid + brightgreen → all valid + """ + if latest_stale: + color, message = "red", "latest stale" + elif stale_count == 0: + color, message = "brightgreen", "all valid" + elif total > 0 and stale_count / total >= 0.5: + color, message = "yellow", f"{stale_count} versions stale" + else: + noun = "version" if stale_count == 1 else "versions" + color, message = "yellowgreen", f"{stale_count} {noun} stale" + return { + "schemaVersion": 1, + "label": "schema deps", + "message": message, + "color": color, + } + + +# --------------------------------------------------------------------------- +# HTML matrix +# --------------------------------------------------------------------------- + +_STATUS_META = { + "valid": ("✅", "#2d8a4e", "Valid"), + "stale": ("⚠️", "#d93f0b", "Stale"), + "no-schema": ("❓", "#9a6700", "Schema not accessible"), + "not-frozen": ("🔄", "#0550ae", "Not frozen"), +} + + +def build_html(results: list[dict], now: str, chemdcatap_url: str, freeze_status: dict) -> str: + rows = [] + + # Dev row — always first, purely informational + rows.append( + "" + "dev" + "latest (floating)" + '🔄 Development — floating import, not checked' + '—' + "" + ) + + for r in results: + ver = r["version"] + ver_label = ( + f"{ver} (latest)" if r["is_latest"] else ver + ) + token = r["token"] or "—" + token_cell = f"{token}" + + icon, color, text = _STATUS_META.get(r["status"], ("❓", "#9a6700", "Unknown")) + + if r["status"] == "stale": + detail = f"dcat-ap-plus {token} not found on GitHub Pages" + status_html = f'{icon} {text} — {detail}' + elif r["status"] == "no-schema": + status_html = ( + f'{icon} {text} — ' + "could not fetch deployed schema" + ) + else: + status_html = f'{icon} {text}' + + fs = freeze_status.get(ver) + if fs is None: + freeze_cell = '—' + elif fs.get("validated"): + freeze_cell = '✅ Validated' + else: + freeze_cell = '❌ Failed' + + rows.append( + f"{ver_label}{token_cell}{status_html}{freeze_cell}" + ) + + rows_html = "\n ".join(rows) + + return f""" + + + + + chem-dcat-ap — Schema Compatibility Matrix + + + +

chem-dcat-ap — Schema Compatibility Matrix

+

+ Last checked: {now}  ·  + Documentation site  ·  + Generated by the check-schema-compatibility workflow +

+ + + + + + + + + + + {rows_html} + +
chem-dcat-ap versiondcat-ap-plus dependencyStatusFreeze validated
+

+ ✅ Valid — the frozen upstream schema URL responds with HTTP 200
+ ⚠️ Stale — the frozen upstream schema URL returns 404 or is unreachable
+ ❓ Schema not accessible — the deployed chem-dcat-ap schema could not be fetched
+ 🔄 Development — not a released version, floating import, not checked
+ ✅/❌ Freeze validated — result of the post-deploy-validate job; — means no data yet +

+ + +""" + + +# --------------------------------------------------------------------------- +# Markdown matrix (MkDocs-compatible) +# --------------------------------------------------------------------------- + +def build_markdown(results: list[dict], now: str, chemdcatap_url: str, freeze_status: dict) -> str: + """Generate a MkDocs-compatible markdown file with the compatibility matrix.""" + rows = [] + + # Dev row — always first, purely informational + rows.append("| `dev` | *latest (floating)* | 🔄 Not frozen / Development — floating import, not checked | — |") + + for r in results: + ver = r["version"] + ver_label = f"**{ver}** *(latest)*" if r["is_latest"] else ver + token = r["token"] or "—" + token_cell = f"`{token}`" + + icon, _, text = _STATUS_META.get(r["status"], ("❓", "#9a6700", "Unknown")) + + if r["status"] == "stale": + status_cell = f"{icon} Stale — dcat-ap-plus `{token}` not found on GitHub Pages" + elif r["status"] == "no-schema": + status_cell = f"{icon} Schema not accessible — could not fetch deployed schema" + elif r["status"] == "not-frozen": + status_cell = f"🔄 Not frozen / Development — floating import, not checked" + else: + status_cell = f"{icon} {text}" + + fs = freeze_status.get(ver) + if fs is None: + freeze_cell = "—" + elif fs.get("validated"): + freeze_cell = "✅ Validated" + else: + freeze_cell = "❌ Failed" + + rows.append(f"| {ver_label} | {token_cell} | {status_cell} | {freeze_cell} |") + + rows_md = "\n".join(rows) + + return f"""--- +title: Schema Compatibility +--- + +# Schema Compatibility Matrix + +Last checked: **{now}** · [Documentation site]({chemdcatap_url}) · Generated by the `check-schema-compatibility` workflow + +| chem-dcat-ap version | dcat-ap-plus dependency | Status | Freeze validated | +|---|---|---|---| +{rows_md} + +**Legend:** + +- ✅ Valid — frozen upstream URL responds HTTP 200 +- ⚠️ Stale — frozen upstream URL returns 404 or is unreachable +- ❓ Schema not accessible — deployed chem-dcat-ap schema could not be fetched +- 🔄 Not frozen / Development — floating import, not checked +- ✅/❌ Freeze validated — result of the post-deploy-validate job; — means no data yet +""" + + +# --------------------------------------------------------------------------- +# Main +# --------------------------------------------------------------------------- + +def main() -> int: + p = argparse.ArgumentParser( + description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter, + ) + p.add_argument( + "--chemdcatap-url", required=True, + help="Base GitHub Pages URL for chem-dcat-ap (no trailing slash)", + ) + p.add_argument( + "--dcatapplus-url", required=True, + help="Base GitHub Pages URL for dcat-ap-plus (no trailing slash)", + ) + p.add_argument( + "--output-dir", required=True, + help="Directory to write badge.json, compatibility.html, compatibility.md, status.env", + ) + args = p.parse_args() + + chemdcatap_base = args.chemdcatap_url.rstrip("/") + dcatapplus_base = args.dcatapplus_url.rstrip("/") + output_dir = Path(args.output_dir) + output_dir.mkdir(parents=True, exist_ok=True) + + # ------------------------------------------------------------------ + # Fetch deployed versions + # ------------------------------------------------------------------ + print(f"Fetching versions from {chemdcatap_base}/versions.json …") + try: + versions_data = fetch_json(f"{chemdcatap_base}/versions.json") + except Exception as exc: + print(f"ERROR: could not fetch versions.json: {exc}", file=sys.stderr) + return 2 + + latest_version = next( + (e["version"] for e in versions_data if "latest" in e.get("aliases", [])), + None, + ) + print(f"Latest released version: {latest_version}") + + # ------------------------------------------------------------------ + # Fetch freeze-validation status + # ------------------------------------------------------------------ + print(f"Fetching freeze-status from {chemdcatap_base}/freeze-status.json …") + freeze_status = fetch_freeze_status(chemdcatap_base) + print(f" Found freeze status for {len(freeze_status)} version(s).") + + # ------------------------------------------------------------------ + # Check each released version (dev excluded) + # ------------------------------------------------------------------ + results: list[dict] = [] + + for entry in versions_data: + version = entry["version"] + if version == "dev": + continue + + is_latest = version == latest_version + schema_url = f"{chemdcatap_base}/{version}/schema/chem_dcat_ap.yaml" + print(f" {version} …", end=" ", flush=True) + + schema_text = fetch_text(schema_url) + if schema_text is None: + print("schema not accessible") + results.append({ + "version": version, "is_latest": is_latest, + "token": None, "status": "no-schema", + }) + continue + + token = extract_dcatapplus_version(schema_text) + if token is None or token == "latest": + print(f"not frozen (token={token!r})") + results.append({ + "version": version, "is_latest": is_latest, + "token": token or "not found", "status": "not-frozen", + }) + continue + + upstream_url = f"{dcatapplus_base}/{token}/schema/dcat_ap_plus.yaml" + ok = url_accessible(upstream_url) + print("✅ valid" if ok else "⚠️ STALE") + results.append({ + "version": version, "is_latest": is_latest, + "token": token, "status": "valid" if ok else "stale", + }) + + # ------------------------------------------------------------------ + # Summary + # ------------------------------------------------------------------ + stale = [r for r in results if r["status"] in ("stale", "no-schema")] + total = len(results) + latest_stale = any(r["is_latest"] and r["status"] != "valid" for r in results) + stale_count = len(stale) + print( + f"\nSummary: {total} released version(s), " + f"{stale_count} stale, latest_stale={latest_stale}" + ) + + # ------------------------------------------------------------------ + # Write badge.json + # ------------------------------------------------------------------ + badge = compute_badge(stale_count, total, latest_stale) + (output_dir / "badge.json").write_text(json.dumps(badge, indent=2) + "\n") + print(f"Badge: {badge['color']} / {badge['message']!r}") + + # ------------------------------------------------------------------ + # Write compatibility.html and compatibility.md + # ------------------------------------------------------------------ + now = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC") + html = build_html(results, now, chemdcatap_base, freeze_status) + (output_dir / "compatibility.html").write_text(html) + print(f"Wrote compatibility.html ({len(html):,} bytes)") + + md = build_markdown(results, now, chemdcatap_base, freeze_status) + (output_dir / "compatibility.md").write_text(md) + print(f"Wrote compatibility.md ({len(md):,} bytes)") + + # ------------------------------------------------------------------ + # Write status.env (KEY=VALUE for GitHub Actions $GITHUB_OUTPUT) + # ------------------------------------------------------------------ + status_lines = [ + f"latest_stale={'true' if latest_stale else 'false'}", + f"stale_count={stale_count}", + f"total_count={total}", + f"badge_color={badge['color']}", + f"badge_message={badge['message']}", + ] + (output_dir / "status.env").write_text("\n".join(status_lines) + "\n") + + if latest_stale: + return 2 + if stale_count > 0: + return 1 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/freeze_imports.py b/scripts/freeze_imports.py new file mode 100644 index 000000000..4fcaa5c62 --- /dev/null +++ b/scripts/freeze_imports.py @@ -0,0 +1,574 @@ +#!/usr/bin/env python3 +"""Freeze LinkML schema imports and identifiers to specific resolved versions. + +This script is called at two points in the release pipeline: + +PHASE 2 (post-deploy-validate job, after gh-pages deploy) +--------------------------------------------------------------------------- +Full freeze — applies ALL transformations at once to the working copy: + + 1. Pin the ``dcatapplus`` prefix value: + dcatapplus: {base}/ -> dcatapplus: {base}/{version}/ + (also handles re-freeze: dcatapplus: {base}/vOLD/ -> {base}/vNEW/) + + 2. Strip any version/alias token from ``dcatapplus:`` import paths: + dcatapplus:latest/schema/X -> dcatapplus:schema/X + + 3. Pin the ``chemdcatap`` prefix value: + chemdcatap: {base}/ -> chemdcatap: {base}/{version}/ + + 4. Convert bare local sub-module imports to ``chemdcatap:schema/`` form: + - chemical_entities_ap -> - chemdcatap:schema/chemical_entities_ap + (enabled via --convert-bare-imports; only safe after Phase 1 deploy + because the versioned CURIE URLs must already exist on gh-pages) + + 5. Remap and version all ``id:`` fields: + id: {old_id_base}/path/ -> id: {new_id_base}/path/{version}/ + + 6. Remap and version own-namespace prefix declarations: + my_ns: {old_id_base}/path/ -> my_ns: {new_id_base}/path/{version}/ + +The modifications are made to the *working copy only* -- they are NOT +committed back to git. Source files stay in their development form on main +(bare imports, unversioned prefixes). Released GitHub Pages snapshots carry +fully versioned, reproducible imports. + +HANDLE-UPSTREAM-RELEASE workflow +--------------------------------------------------------------------------- +Only steps 1-2 are invoked (with --dcatapplus-base + --dcatapplus-version, +no --convert-bare-imports, no --schema-id-*) to pin dcatapplus on main. + +Usage examples +-------------- +Full post-deploy freeze (Phase 2) — production: + + uv run python scripts/freeze_imports.py \\ + --schema-dir src/chem_dcat_ap/schema \\ + --dcatapplus-base https://w3id.org/nfdi-de/dcat-ap-plus \\ + --versions-url https://nfdi-de.github.io/dcat-ap-plus/versions.json \\ + --chemdcatap-base https://nfdi-de.github.io/chem-dcat-ap/chemistry \\ + --chemdcatap-version v0.5.0 \\ + --convert-bare-imports \\ + --schema-id-old-base https://w3id.org/nfdi-de/dcat-ap-plus \\ + --schema-id-new-base https://w3id.org/nfdi-de/dcat-ap-plus + +Upstream-release pin only (handle-upstream-release workflow): + + uv run python scripts/freeze_imports.py \\ + --schema-dir src/chem_dcat_ap/schema \\ + --dcatapplus-base https://w3id.org/nfdi-de/dcat-ap-plus \\ + --dcatapplus-version v0.5.0 + +Exit codes +---------- +0 Success. +1 Error -- details on stderr. +""" + +import argparse +import json +import re +import sys +import urllib.request +from pathlib import Path + + +# --------------------------------------------------------------------------- +# Core helpers +# --------------------------------------------------------------------------- + +def resolve_alias(versions_url: str, alias: str = "latest") -> str: + """Return the version string that currently carries *alias* in mike's versions.json.""" + try: + with urllib.request.urlopen(versions_url, timeout=15) as resp: + versions = json.load(resp) + except Exception as exc: + raise RuntimeError(f"Could not fetch {versions_url}: {exc}") from exc + + for entry in versions: + if alias in entry.get("aliases", []): + return entry["version"] + + available = [e.get("version") for e in versions] + raise ValueError( + f"Alias '{alias}' not found in {versions_url}. " + f"Available versions: {available}" + ) + + +def freeze_dcatapplus_prefix(text: str, base: str, version: str) -> str: + """Pin the dcatapplus prefix value: base/ -> base/version/ + + Replaces lines like: + dcatapplus: https://w3id.org/nfdi-de/dcat-ap-plus/ + with: + dcatapplus: https://w3id.org/nfdi-de/dcat-ap-plus/v0.3.0/ + + Also handles re-freeze when the prefix is already pinned to a previous + version, e.g.: + dcatapplus: https://w3id.org/nfdi-de/dcat-ap-plus/v0.3.0/ + -> + dcatapplus: https://w3id.org/nfdi-de/dcat-ap-plus/v0.4.0/ + """ + new = f"dcatapplus: {base}/{version}/\n" + # First try exact match on the base form (most common during first freeze) + old_base = f"dcatapplus: {base}/\n" + if old_base in text: + return text.replace(old_base, new) + # Fall back: replace any already-versioned form dcatapplus: {base}/{token}/ + return re.sub( + r"dcatapplus: " + re.escape(base) + r"/[^/\s]+/\n", + new, + text, + ) + + +def strip_dcatapplus_import_path_token(text: str) -> str: + """Remove any version/alias token from dcatapplus import paths. + + Handles patterns like: + - dcatapplus:latest/schema/foo -> dcatapplus:schema/foo + - dcatapplus:v0.3.0/schema/foo -> dcatapplus:schema/foo + + Only matches when there is a token (non-colon, non-slash chars) between + 'dcatapplus:' and '/schema/' -- does not touch lines already in the + 'dcatapplus:schema/' form. + """ + return re.sub(r"dcatapplus:[^:/\s]+/schema/", "dcatapplus:schema/", text) + + +def freeze_chemdcatap_prefix(text: str, base: str, version: str) -> str: + """Pin the chemdcatap prefix value: base/ -> base/version/ + + Replaces lines like: + chemdcatap: https://nfdi-de.github.io/chem-dcat-ap/chemistry/ + with: + chemdcatap: https://nfdi-de.github.io/chem-dcat-ap/chemistry/v0.2.0/ + + Also handles re-freeze from a previously versioned value. + """ + new = f"chemdcatap: {base}/{version}/\n" + old_base = f"chemdcatap: {base}/\n" + if old_base in text: + return text.replace(old_base, new) + # Fallback: replace already-versioned form + return re.sub( + r"chemdcatap: " + re.escape(base) + r"/[^/\s]+/\n", + new, + text, + ) + + +def convert_bare_imports(text: str) -> str: + """Convert bare local sub-module imports to prefixed CURIE form. + + For each bare import name, checks whether a matching named prefix is + declared in the schema. If so, uses that prefix as the CURIE namespace: + - chemical_entities_ap -> - chemical_entities_ap:schema/chemical_entities_ap + + If no matching prefix is declared, falls back to the chemdcatap: prefix: + - chemical_entities_ap -> - chemdcatap:schema/chemical_entities_ap + + This handles two deployment scenarios: + + - Production (w3id.org): each sub-schema has its own prefix pointing to + a distinct w3id path (chemistry/entity/, chemistry/reaction/, etc.). + Using that prefix ensures the CURIE resolves to the correct w3id URL. + The sub-module prefix declarations must exist in the source schema: + chemical_entities_ap: https://w3id.org/nfdi-de/dcat-ap-plus/chemistry/entity/ + + - Test repo: all sub-schemas are deployed under a single chemdcatap: path, + so no individual prefix is declared and the chemdcatap: fallback is used. + + Only items inside the top-level ``imports:`` block are converted. + Class-level slot lists, mixin lists, etc. are left untouched. + Already-prefixed imports (containing ':') are never touched. + + NOTE: only safe to run AFTER the schemas have been deployed to gh-pages, + because the resolved CURIE URLs must already exist at the versioned path. + Enable via --convert-bare-imports. + """ + # Collect all declared prefix names so we can pick the right CURIE namespace + # for each bare import (two-space-indented prefix declarations). + declared_prefixes = set(re.findall(r'^ ([A-Za-z][A-Za-z0-9_-]*):', text, re.MULTILINE)) + + lines = text.splitlines(keepends=True) + result = [] + in_imports_block = False + for line in lines: + stripped = line.rstrip() + + # Detect the start of the top-level imports: block. + if re.match(r'^imports:\s*$', stripped): + in_imports_block = True + result.append(line) + continue + + # Any new top-level key (no leading whitespace, not a comment, not a + # bare list item) signals the end of the imports block. + if stripped and stripped[0] not in (' ', '\t', '#') and not stripped.startswith('-'): + in_imports_block = False + + if in_imports_block: + m = re.match(r'^(\s+- )([A-Za-z][A-Za-z0-9_-]+)\s*$', line) + if m and ':' not in m.group(2) and '/' not in m.group(2): + name = m.group(2) + prefix = name if name in declared_prefixes else "chemdcatap" + result.append(f"{m.group(1)}{prefix}:schema/{name}\n") + continue + + result.append(line) + return "".join(result) + + +def freeze_schema_id(text: str, old_base: str, new_base: str, version: str) -> str: + """Remap and version the ``id:`` field of a schema. + + Replaces any line of the form: + id: {old_base}{suffix}/ + + with: + id: {new_base}{suffix}/{version}/ + + For example, with old_base=new_base=https://w3id.org/nfdi-de/dcat-ap-plus, + version=v0.5.0: + + id: https://w3id.org/nfdi-de/dcat-ap-plus/chemistry/ + -> id: https://w3id.org/nfdi-de/dcat-ap-plus/chemistry/v0.5.0/ + + id: https://w3id.org/nfdi-de/dcat-ap-plus/chemistry/entity/ + -> id: https://w3id.org/nfdi-de/dcat-ap-plus/chemistry/entity/v0.5.0/ + + When old_base == new_base (standard production case) the function simply + inserts the version token without changing the base URL. + """ + def _replace(m: re.Match) -> str: + suffix = m.group(1).rstrip("/") # e.g. "/chemistry" or "/chemistry/entity" + return f"id: {new_base}{suffix}/{version}/" + + return re.sub( + r"^id: " + re.escape(old_base) + r"(/[^\n]*/)$", + _replace, + text, + flags=re.MULTILINE, + ) + + +def freeze_own_namespace_prefixes( + text: str, old_base: str, new_base: str, version: str +) -> str: + """Remap and version sub-schema own-namespace prefix declarations. + + Matches any indented prefix line (two leading spaces) whose value + starts with ``old_base`` and ends with ``/``, EXCEPT the reserved + names ``chemdcatap`` and ``dcatapplus`` (handled separately). + + For example (production, old_base == new_base == w3id.org base): + chemical_entities_ap: https://w3id.org/nfdi-de/dcat-ap-plus/chemistry/entity/ + -> chemical_entities_ap: https://w3id.org/nfdi-de/dcat-ap-plus/chemistry/entity/v0.5.0/ + + material_entities_ap: https://w3id.org/nfdi-de/dcat-ap-plus/materials/ + -> material_entities_ap: https://w3id.org/nfdi-de/dcat-ap-plus/materials/v0.5.0/ + """ + def _replace(m: re.Match) -> str: + prefix_name = m.group(1) + suffix = m.group(2).rstrip("/") # e.g. "/chemistry/entity" + return f" {prefix_name}: {new_base}{suffix}/{version}/" + + # Exclude chemdcatap and dcatapplus — they are frozen by their own functions + return re.sub( + r"^ (?!chemdcatap|dcatapplus)([A-Za-z][A-Za-z0-9_-]*): " + + re.escape(old_base) + + r"(/[^\n]*/)$", + _replace, + text, + flags=re.MULTILINE, + ) + + +def process_file( + yaml_file: Path, + dcatapplus_base: str | None, + dcatapplus_version: str | None, + chemdcatap_base: str | None, + chemdcatap_version: str | None, + convert_bare: bool, + schema_id_old_base: str | None, + schema_id_new_base: str | None, + sub_module_base: str | None = None, +) -> bool: + """Apply all requested freezes to one file. Returns True if the file changed.""" + text = yaml_file.read_text(encoding="utf-8") + original = text + + # 1. Pin dcatapplus prefix + strip import path token + if dcatapplus_base and dcatapplus_version: + text = freeze_dcatapplus_prefix(text, dcatapplus_base, dcatapplus_version) + text = strip_dcatapplus_import_path_token(text) + + # 2. Pin chemdcatap prefix + if chemdcatap_base and chemdcatap_version: + text = freeze_chemdcatap_prefix(text, chemdcatap_base, chemdcatap_version) + + # 3. Convert bare local imports to prefixed CURIE form. + # Uses each module's own declared prefix when available; falls back to + # chemdcatap: for modules without an individual prefix declaration. + # Only safe after Phase 1 deploy (versioned URLs must already exist). + if convert_bare and chemdcatap_base and chemdcatap_version: + text = convert_bare_imports(text) + + # 4. Remap + version id: field + if schema_id_old_base and schema_id_new_base and chemdcatap_version: + text = freeze_schema_id( + text, schema_id_old_base, schema_id_new_base, chemdcatap_version + ) + + # 5. Version own-namespace prefix declarations. + # Two independent code paths: + # a) --sub-module-base: versions prefixes matching the given base WITHOUT + # touching id: fields. Use in production where sub-schemas have distinct + # w3id paths (chemistry/entity/, chemistry/reaction/, materials/, etc.). + # b) --schema-id-*: also remaps the base URL in id: fields. Use in the test + # repo where id: fields must be redirected to the test GitHub Pages URL. + if sub_module_base and chemdcatap_version: + text = freeze_own_namespace_prefixes( + text, sub_module_base, sub_module_base, chemdcatap_version + ) + elif schema_id_old_base and schema_id_new_base and chemdcatap_version: + text = freeze_own_namespace_prefixes( + text, schema_id_old_base, schema_id_new_base, chemdcatap_version + ) + + if text != original: + yaml_file.write_text(text, encoding="utf-8") + return True + return False + + +# --------------------------------------------------------------------------- +# CLI +# --------------------------------------------------------------------------- + +def build_parser() -> argparse.ArgumentParser: + p = argparse.ArgumentParser( + description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter, + ) + p.add_argument( + "--schema-dir", + required=True, + help="Directory containing the schema YAML files to update.", + ) + # --- dcatapplus --- + p.add_argument( + "--dcatapplus-base", + default=None, + help=( + "Base URL for the dcatapplus prefix, no trailing slash. " + "e.g. https://w3id.org/nfdi-de/dcat-ap-plus" + ), + ) + p.add_argument( + "--dcatapplus-version", + default=None, + help=( + "Explicit version to pin dcatapplus to (e.g. 'v0.3.0'). " + "If omitted and --versions-url is given, the 'latest' alias is resolved." + ), + ) + p.add_argument( + "--versions-url", + default=None, + help=( + "URL to the mike versions.json of the upstream dcat-ap-plus repo. " + "Used to auto-resolve the 'latest' alias when --dcatapplus-version " + "is not given." + ), + ) + # --- chemdcatap --- + p.add_argument( + "--chemdcatap-base", + default=None, + help=( + "Base URL for the chemdcatap prefix, no trailing slash. " + "e.g. https://nfdi-de.github.io/chem-dcat-ap/chemistry" + ), + ) + p.add_argument( + "--chemdcatap-version", + default=None, + help="Version to pin chemdcatap to (e.g. the current release tag 'v0.2.0').", + ) + # --- bare import conversion --- + p.add_argument( + "--convert-bare-imports", + action="store_true", + default=False, + help=( + "Convert bare local sub-module imports to chemdcatap:schema/ CURIE form. " + "Only safe to run AFTER the schemas have been deployed to gh-pages " + "(Phase 2 / post-deploy-validate job), because the CURIE URLs must " + "already exist at the versioned path." + ), + ) + # --- sub-module prefix versioning --- + p.add_argument( + "--sub-module-base", + default=None, + help=( + "Base URL for versioning sub-module prefix declarations, no trailing slash. " + "e.g. https://w3id.org/nfdi-de/dcat-ap-plus " + "Versions any indented prefix whose value starts with this base: " + " name: {base}/path/ -> name: {base}/path/{version}/ " + "using --chemdcatap-version as the version token. " + "Use in production repos where sub-schemas have individual w3id paths " + "(chemistry/entity/, chemistry/reaction/, materials/, etc.). " + "Requires --chemdcatap-version. Does NOT affect id: fields " + "(contrast with --schema-id-old-base which also remaps id: fields)." + ), + ) + # --- id: and own-namespace remapping --- + p.add_argument( + "--schema-id-old-base", + default=None, + help=( + "Base URL currently used in schema id: fields and own-namespace prefixes. " + "e.g. https://w3id.org/nfdi-de/dcat-ap-plus " + "Will be replaced with --schema-id-new-base and the version injected." + ), + ) + p.add_argument( + "--schema-id-new-base", + default=None, + help=( + "Replacement base URL for schema id: fields and own-namespace prefixes. " + "For production use, set this to the same value as --schema-id-old-base " + "(w3id.org) so only the version is injected without changing the base URL." + ), + ) + return p + + +def main() -> int: + args = build_parser().parse_args() + + schema_dir = Path(args.schema_dir) + if not schema_dir.is_dir(): + print(f"ERROR: not a directory: {schema_dir}", file=sys.stderr) + return 1 + + # ------------------------------------------------------------------ + # Validate argument combinations + # ------------------------------------------------------------------ + if args.schema_id_old_base and not args.schema_id_new_base: + print( + "ERROR: --schema-id-old-base requires --schema-id-new-base.", + file=sys.stderr, + ) + return 1 + if args.schema_id_new_base and not args.schema_id_old_base: + print( + "ERROR: --schema-id-new-base requires --schema-id-old-base.", + file=sys.stderr, + ) + return 1 + if args.schema_id_old_base and not args.chemdcatap_version: + print( + "ERROR: --schema-id-old-base requires --chemdcatap-version " + "(used as the version token for id: and namespace prefix freeze).", + file=sys.stderr, + ) + return 1 + if args.convert_bare_imports and not (args.chemdcatap_base and args.chemdcatap_version): + print( + "ERROR: --convert-bare-imports requires --chemdcatap-base and " + "--chemdcatap-version (needed to version the chemdcatap: fallback prefix).", + file=sys.stderr, + ) + return 1 + if args.sub_module_base and not args.chemdcatap_version: + print( + "ERROR: --sub-module-base requires --chemdcatap-version " + "(used as the version token for sub-module prefix versioning).", + file=sys.stderr, + ) + return 1 + + # ------------------------------------------------------------------ + # Resolve dcatapplus version + # ------------------------------------------------------------------ + dcatapplus_version: str | None = None + if args.dcatapplus_base: + if args.dcatapplus_version: + dcatapplus_version = args.dcatapplus_version + elif args.versions_url: + print(f"Resolving 'latest' alias from: {args.versions_url}") + try: + dcatapplus_version = resolve_alias(args.versions_url, "latest") + except Exception as exc: + print(f"ERROR: {exc}", file=sys.stderr) + return 1 + print(f" -> resolved to: {dcatapplus_version}") + else: + print( + "ERROR: --dcatapplus-base given but neither --dcatapplus-version " + "nor --versions-url was provided.", + file=sys.stderr, + ) + return 1 + + chemdcatap_version: str | None = args.chemdcatap_version + + # Require at least one operation + has_work = ( + bool(dcatapplus_version) + or bool(chemdcatap_version) + or args.convert_bare_imports + or bool(args.schema_id_old_base) + or bool(args.sub_module_base) + ) + if not has_work: + print( + "ERROR: nothing to do -- provide at least one of: " + "--dcatapplus-version (or --versions-url), --chemdcatap-version, " + "--convert-bare-imports, --schema-id-old-base.", + file=sys.stderr, + ) + return 1 + + # ------------------------------------------------------------------ + # Process all YAML files in schema_dir + # ------------------------------------------------------------------ + changed_files: list[str] = [] + for yaml_file in sorted(schema_dir.glob("*.yaml")): + if process_file( + yaml_file, + dcatapplus_base=args.dcatapplus_base, + dcatapplus_version=dcatapplus_version, + chemdcatap_base=args.chemdcatap_base, + chemdcatap_version=chemdcatap_version, + convert_bare=args.convert_bare_imports, + schema_id_old_base=args.schema_id_old_base, + schema_id_new_base=args.schema_id_new_base, + sub_module_base=args.sub_module_base, + ): + changed_files.append(yaml_file.name) + + if changed_files: + print(f"Modified files: {', '.join(changed_files)}") + else: + print(f"No files changed in {schema_dir}") + + # ------------------------------------------------------------------ + # Emit machine-parseable KEY=VALUE lines for shell callers + # ------------------------------------------------------------------ + if dcatapplus_version: + print(f"FROZEN_VERSION={dcatapplus_version}") + if chemdcatap_version: + print(f"FROZEN_CHEMDCATAP_VERSION={chemdcatap_version}") + + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/should_update_latest.py b/scripts/should_update_latest.py new file mode 100644 index 000000000..6998e39b2 --- /dev/null +++ b/scripts/should_update_latest.py @@ -0,0 +1,105 @@ +#!/usr/bin/env python3 +"""Decide whether a newly pushed tag should replace the 'latest' mike alias. + +Compares the new version against the version currently carrying the 'latest' +alias in this repo's mike versions.json. Outputs exactly one line: + + true — new_version >= current_latest (promote 'latest') + false — new_version < current_latest (keep existing 'latest') + +This prevents a maintenance release on an older major branch (e.g. v1.10.4) +from overwriting the 'latest' alias when a newer major (e.g. v2.0.1) is +already live. + +On any error (network failure, unparseable version, etc.) the script defaults +to "true" so a release is never silently skipped. + +Usage: + uv run python scripts/should_update_latest.py \\ + --new-version v0.2.0 \\ + --versions-url https://HendrikBorgelt.github.io/test_chemDCAT_ap_versioning_freeze/versions.json + +Exit codes +---------- +0 Always (the answer is encoded in stdout). +""" + +import argparse +import json +import sys +import urllib.request + +from packaging.version import InvalidVersion, Version + + +# --------------------------------------------------------------------------- +# Core helper +# --------------------------------------------------------------------------- + +def should_update(new_version_str: str, versions_url: str) -> bool: + """Return True if new_version_str should become the new 'latest'.""" + try: + with urllib.request.urlopen(versions_url, timeout=15) as resp: + versions = json.load(resp) + except Exception as exc: + print( + f"WARNING: Could not fetch {versions_url} ({exc}). " + "Defaulting to update latest.", + file=sys.stderr, + ) + return True + + current_latest_str = next( + (e["version"] for e in versions if "latest" in e.get("aliases", [])), + None, + ) + + if current_latest_str is None: + # No 'latest' alias exists yet — always promote + return True + + try: + new_v = Version(new_version_str.lstrip("v")) + cur_v = Version(current_latest_str.lstrip("v")) + except InvalidVersion as exc: + print( + f"WARNING: Could not parse version for comparison ({exc}). " + "Defaulting to update latest.", + file=sys.stderr, + ) + return True + + return new_v >= cur_v + + +# --------------------------------------------------------------------------- +# CLI +# --------------------------------------------------------------------------- + +def build_parser() -> argparse.ArgumentParser: + p = argparse.ArgumentParser( + description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter, + ) + p.add_argument( + "--new-version", + required=True, + help="The version tag just pushed, e.g. 'v0.2.0'.", + ) + p.add_argument( + "--versions-url", + required=True, + help="URL to *this* repo's mike versions.json.", + ) + return p + + +def main() -> int: + args = build_parser().parse_args() + result = should_update(args.new_version, args.versions_url) + print("true" if result else "false") + return 0 + + +if __name__ == "__main__": + sys.exit(main())