Summary
Releasing v5.4.0 (PR #823) completed with every job green, but the legacy aiox-core npm package — the one users actually install (npm install -g aiox-core, npx aiox-core install) — was never republished. No red job, no error anywhere. Three distinct, compounding bugs in the same release chain were found and fixed live while investigating; documenting all of them here so they don't recur on the next release.
Bug 1 — lockstep version sync is never persisted to main
Symptom: npm view @aiox-squads/core version → 5.4.0 (correct, has the fix). npm view aiox-core version → 5.3.0 (stale, does not have the fix), with dependencies["@aiox-squads/core"] pinned exactly to 5.3.0.
Cause: .releaserc.json runs @semantic-release/exec → scripts/sync-version-lockstep.js in its prepareCmd, which updates .aiox-core/package.json and compat/aiox-core/package.json to the new version — but this only happens inside the Semantic Release job's own ephemeral checkout. There is no @semantic-release/git plugin, so the bump is never committed/pushed back to main. The downstream publish_legacy_aiox_core job runs in a separate workflow (npm-publish.yml, chained via gh workflow run dispatch) with its own fresh checkout of main, reads the still-5.3.0 compat/aiox-core/package.json, concludes "Version 5.3.0 already exists" → should_publish=false → silently skips.
Evidence:
Semantic Release run 31896753245 — log shows "The next release version is 5.4.0" and successful publish of @aiox-squads/core@5.4.0.
- Chained
npm-publish.yml run 31896804383 — publish_legacy_aiox_core job log: "Version 5.3.0 already exists for aiox-core" → skipped.
git show origin/main:compat/aiox-core/package.json (before fix) showed 5.3.0 despite the release having already shipped 5.4.0.
Fixed by: PR #825 — ran node scripts/sync-version-lockstep.js 5.4.0 (the pipeline's own tool, for byte-identical output) and committed .aiox-core/package.json + compat/aiox-core/package.json directly to main.
Bug 2 — the lockstep set is incomplete: root package.json isn't covered
Symptom: After fixing Bug 1 (PR #825), CI on main broke: tests/cli/validate-publish.test.js started failing. node bin/utils/validate-publish.js locally: "FAIL: version drift: .aiox-core/package.json version 5.4.0 does not match root package.json version 5.3.0. They must move in lockstep — root is SOT."
Cause: scripts/sync-version-lockstep.js treats root package.json as a read-only reference (it reads rootPkg.version to know the target, but never writes to root itself) — this is by design, documented in the script's own header comment. But root package.json is also never bumped anywhere else (same root cause as Bug 1: no @semantic-release/git plugin persists @semantic-release/npm's local version bump back to main). Before PR #825, root and .aiox-core/package.json were both stale at 5.3.0 — inconsistent with the tag, but internally consistent with each other, so the safety gate passed. PR #825 bumped only one side of that pairing, introducing a new drift that broke validate-aiox-core-namespace.js's root-as-SOT invariant. This was caught by the PR's own CI (Jest Tests, Validation Summary all red) but merged anyway with --admin before checking gh pr checks.
Fixed by: PR #826 — bumped root package.json to 5.4.0 (matching the already-published @aiox-squads/core@5.4.0 and the already-pushed v5.4.0 tag), synced package-lock.json's two version fields via npm install --package-lock-only, and regenerated .aiox-core/install-manifest.yaml. Verified locally (validate-publish.js → PASS, the specific Jest suite → 15/15) before pushing, and waited for full green CI before the --admin merge this time.
Bug 3 — publish_legacy_aiox_core's own if: never actually allows a scoped packages=aiox-core dispatch
Symptom: Re-dispatching npm-publish.yml -f packages=aiox-core (intending to publish only the legacy wrapper, without re-touching @aiox-squads/core) resulted in both publish and publish_legacy_aiox_core showing conclusion: skipped — nothing published.
Cause:
publish_legacy_aiox_core:
needs: [test, build, publish]
if: ${{ contains(...,'aiox-core,') && (needs.publish.result == 'success' || needs.publish.result == 'skipped') }}
The condition explicitly tries to allow the needs.publish.result == 'skipped' case (expected when only aiox-core — not core — is selected, since publish's own if: only runs on contains(packages, ',core,')). But GitHub Actions' default behavior overrides this: if a job in needs is skipped, the dependent job is also skipped by default, regardless of a custom if: expression, unless that expression is wrapped in always(). Since this if: doesn't call always(), publish_legacy_aiox_core can never run when publish was skipped — making a scoped packages=aiox-core-only dispatch a dead combination. It only worked (accidentally) in earlier attempts because packages=all was used, which makes publish run for real (not skip) and satisfies GitHub's default propagation rule as a side effect.
Workaround used today: dispatched with packages=all instead of packages=aiox-core. Safe because publish has its own internal idempotent "already published" check (same pattern seen for @aiox-squads/installer and aiox-core) — verified: it correctly no-op'd for @aiox-squads/core@5.4.0 (npm view @aiox-squads/core version unchanged at 5.4.0 after the run).
Suggested fix (not applied — flagging for review): add always() to the front of publish_legacy_aiox_core's if: (and audit the other conditional jobs in this workflow for the same pattern), so a scoped packages=aiox-core dispatch actually works as documented/intended.
Impact
Every future release using this chain repeats Bug 1/2 unless someone manually notices and intervenes (as happened here, live, mid-release). The scoped fix (Bug 3) means operators can't cleanly republish just the legacy wrapper without also re-touching the scoped package's publish job. In the concrete case that triggered all this: a student with a Pro-activation failure would have remained on a broken aiox-core@5.3.0 even after a "successful" v5.4.0 release, until this was manually caught and fixed today (PRs #825, #826, plus a manual packages=all re-dispatch of npm-publish.yml).
Secondary finding: Cross-Platform Tests (windows-latest) has been red on every push to main since at least 2026-07-10
Unrelated to the above, but noticed while confirming main's post-merge status: the push-triggered CI workflow's Cross-Platform Tests (windows-latest, Node 18/20/22/24) jobs have failed on every push-to-main run checked going back to commit 0ad5a9f3 (2026-07-10) — predates all of today's changes. This job doesn't run on pull_request events (shows skipping there), so it's invisible in normal PR review and has presumably been silently red for over a month.
Suggested remediation options (not prescribing — flagging for a maintainer decision)
- Bug 1/2: either add a scoped
@semantic-release/git plugin (committing just the version-bearing files back to main after a successful release), or have the release job directly trigger npm-publish.yml with an explicit version input instead of having downstream jobs re-derive it from a fresh (and potentially stale) checkout of main.
- Bug 3: add
always() to publish_legacy_aiox_core's if: condition (and audit sibling jobs for the same gap).
- Process hygiene:
should_publish=false / job-skipped states in this chain should never render as a fully green run without an explicit summary callout — worth a ::warning:: annotation at minimum.
- Cross-Platform Tests: separately worth its own triage — it's been broken long enough that it's providing zero signal today.
Labels
Suggest: bug, infra, release, ci, high priority (this directly caused a real user-facing regression to go unpatched after a "successful" release).
Summary
Releasing v5.4.0 (PR #823) completed with every job green, but the legacy
aiox-corenpm package — the one users actually install (npm install -g aiox-core,npx aiox-core install) — was never republished. No red job, no error anywhere. Three distinct, compounding bugs in the same release chain were found and fixed live while investigating; documenting all of them here so they don't recur on the next release.Bug 1 — lockstep version sync is never persisted to
mainSymptom:
npm view @aiox-squads/core version→5.4.0(correct, has the fix).npm view aiox-core version→5.3.0(stale, does not have the fix), withdependencies["@aiox-squads/core"]pinned exactly to5.3.0.Cause:
.releaserc.jsonruns@semantic-release/exec→scripts/sync-version-lockstep.jsin itsprepareCmd, which updates.aiox-core/package.jsonandcompat/aiox-core/package.jsonto the new version — but this only happens inside theSemantic Releasejob's own ephemeral checkout. There is no@semantic-release/gitplugin, so the bump is never committed/pushed back tomain. The downstreampublish_legacy_aiox_corejob runs in a separate workflow (npm-publish.yml, chained viagh workflow rundispatch) with its own fresh checkout ofmain, reads the still-5.3.0compat/aiox-core/package.json, concludes "Version 5.3.0 already exists" →should_publish=false→ silently skips.Evidence:
Semantic Releaserun31896753245— log shows "The next release version is 5.4.0" and successful publish of@aiox-squads/core@5.4.0.npm-publish.ymlrun31896804383—publish_legacy_aiox_corejob log: "Version 5.3.0 already exists for aiox-core" → skipped.git show origin/main:compat/aiox-core/package.json(before fix) showed5.3.0despite the release having already shipped5.4.0.Fixed by: PR #825 — ran
node scripts/sync-version-lockstep.js 5.4.0(the pipeline's own tool, for byte-identical output) and committed.aiox-core/package.json+compat/aiox-core/package.jsondirectly tomain.Bug 2 — the lockstep set is incomplete: root
package.jsonisn't coveredSymptom: After fixing Bug 1 (PR #825), CI on
mainbroke:tests/cli/validate-publish.test.jsstarted failing.node bin/utils/validate-publish.jslocally: "FAIL: version drift: .aiox-core/package.json version 5.4.0 does not match root package.json version 5.3.0. They must move in lockstep — root is SOT."Cause:
scripts/sync-version-lockstep.jstreats rootpackage.jsonas a read-only reference (it readsrootPkg.versionto know the target, but never writes to root itself) — this is by design, documented in the script's own header comment. But rootpackage.jsonis also never bumped anywhere else (same root cause as Bug 1: no@semantic-release/gitplugin persists@semantic-release/npm's local version bump back tomain). Before PR #825, root and.aiox-core/package.jsonwere both stale at5.3.0— inconsistent with the tag, but internally consistent with each other, so the safety gate passed. PR #825 bumped only one side of that pairing, introducing a new drift that brokevalidate-aiox-core-namespace.js's root-as-SOT invariant. This was caught by the PR's own CI (Jest Tests, Validation Summary all red) but merged anyway with--adminbefore checkinggh pr checks.Fixed by: PR #826 — bumped root
package.jsonto5.4.0(matching the already-published@aiox-squads/core@5.4.0and the already-pushedv5.4.0tag), syncedpackage-lock.json's two version fields vianpm install --package-lock-only, and regenerated.aiox-core/install-manifest.yaml. Verified locally (validate-publish.js→ PASS, the specific Jest suite → 15/15) before pushing, and waited for full green CI before the--adminmerge this time.Bug 3 —
publish_legacy_aiox_core's ownif:never actually allows a scopedpackages=aiox-coredispatchSymptom: Re-dispatching
npm-publish.yml -f packages=aiox-core(intending to publish only the legacy wrapper, without re-touching@aiox-squads/core) resulted in bothpublishandpublish_legacy_aiox_coreshowingconclusion: skipped— nothing published.Cause:
The condition explicitly tries to allow the
needs.publish.result == 'skipped'case (expected when onlyaiox-core— notcore— is selected, sincepublish's ownif:only runs oncontains(packages, ',core,')). But GitHub Actions' default behavior overrides this: if a job inneedsis skipped, the dependent job is also skipped by default, regardless of a customif:expression, unless that expression is wrapped inalways(). Since thisif:doesn't callalways(),publish_legacy_aiox_corecan never run whenpublishwas skipped — making a scopedpackages=aiox-core-only dispatch a dead combination. It only worked (accidentally) in earlier attempts becausepackages=allwas used, which makespublishrun for real (not skip) and satisfies GitHub's default propagation rule as a side effect.Workaround used today: dispatched with
packages=allinstead ofpackages=aiox-core. Safe becausepublishhas its own internal idempotent "already published" check (same pattern seen for@aiox-squads/installerandaiox-core) — verified: it correctly no-op'd for@aiox-squads/core@5.4.0(npm view @aiox-squads/core versionunchanged at5.4.0after the run).Suggested fix (not applied — flagging for review): add
always()to the front ofpublish_legacy_aiox_core'sif:(and audit the other conditional jobs in this workflow for the same pattern), so a scopedpackages=aiox-coredispatch actually works as documented/intended.Impact
Every future release using this chain repeats Bug 1/2 unless someone manually notices and intervenes (as happened here, live, mid-release). The scoped fix (Bug 3) means operators can't cleanly republish just the legacy wrapper without also re-touching the scoped package's publish job. In the concrete case that triggered all this: a student with a Pro-activation failure would have remained on a broken
aiox-core@5.3.0even after a "successful" v5.4.0 release, until this was manually caught and fixed today (PRs #825, #826, plus a manualpackages=allre-dispatch ofnpm-publish.yml).Secondary finding:
Cross-Platform Tests (windows-latest)has been red on every push tomainsince at least 2026-07-10Unrelated to the above, but noticed while confirming
main's post-merge status: thepush-triggeredCIworkflow'sCross-Platform Tests (windows-latest, Node 18/20/22/24)jobs have failed on every push-to-main run checked going back to commit0ad5a9f3(2026-07-10) — predates all of today's changes. This job doesn't run onpull_requestevents (showsskippingthere), so it's invisible in normal PR review and has presumably been silently red for over a month.Suggested remediation options (not prescribing — flagging for a maintainer decision)
@semantic-release/gitplugin (committing just the version-bearing files back tomainafter a successful release), or have the release job directly triggernpm-publish.ymlwith an explicitversioninput instead of having downstream jobs re-derive it from a fresh (and potentially stale) checkout ofmain.always()topublish_legacy_aiox_core'sif:condition (and audit sibling jobs for the same gap).should_publish=false/ job-skipped states in this chain should never render as a fully green run without an explicit summary callout — worth a::warning::annotation at minimum.Labels
Suggest:
bug,infra,release,ci, high priority (this directly caused a real user-facing regression to go unpatched after a "successful" release).