Skip to content

harness canary: the scheduled run resolves matching drift issues (no dispatch token) - #52

Merged
davidcrowe merged 1 commit into
mainfrom
feat/canary-sweeps-drift
Oct 8, 2026
Merged

davidcrowe merged 1 commit into
mainfrom
feat/canary-sweeps-drift

Conversation

@davidcrowe

Copy link
Copy Markdown
Contributor

Founder decision: no CANARY_DISPATCH_TOKEN. Instead of the drift scout triggering the canary, the scheduled canary (every 6h, @latest) finds and resolves the scout's release issues itself.

What changes

scripts/canary-report.mjs

  • After reporting the leg, lists open drift issues in davidcrowe/gatewaystack-connect, keeps those whose body carries <!-- drift-scout:key=release:<entryId>@<version> --> for an entry mapped to this leg (LEG_ENTRIES mirrors canaryLeg in the scout's registry.ts: 11 harnesses by name, crewai -> sdk-crewai, langchain-core + langgraph -> sdk-langchain, anthropic-sdk-typescript -> sdk-governance-anthropic) and whose normalised version (same normalizeVersion rule as the scout's issue.ts) equals the version this run tested.
  • Pass: comments Canary <leg> on <version>: ✅ passed — closing. and closes (state_reason: completed).
  • Fail: comments Canary <leg> on <version>: ❌ failed at <step>: <canary issue / run link> once per failure; a hidden <!-- harness-canary:verdict leg=.. version=.. result=.. --> marker lets a later run see that its last verdict on that issue already says failed for the same version and not repeat it.
  • Docs-type drift issues (non-release: markers) are never touched. Best effort: a sweep error is logged, never fails the reporter.
  • --drift-issue pairing path (harness canary: post the verdict on the drift issue that dispatched the run #51) unchanged; the sweep skips that issue so it is not commented twice.
  • New --tested entryId=version,... for SDK legs (their --version is the ACP package's, not the upstream's); an SDK entry with no pair is skipped. New --fixture <json> for --dry-run previews.

.github/workflows/harness-canary.yml

  • Harness legs already pass the resolved harness version. SDK job: install-package now also records the upstream versions it installed (langchain-core, langgraph, crewai, @anthropic-ai/sdk) as an upstream output, passed to the reporter as --tested.
  • Header comment updated; repository_dispatch kept for hand-triggered runs.

docs/harness-canary.md: "Drift pairing" replaced by "Drift sweep": the scheduled canary resolves matching drift issues within ~6h; no extra token. CANARY_DISPATCH_TOKEN instructions removed.

Validation

  • npx --yes js-yaml .github/workflows/harness-canary.yml OK; node --check scripts/canary-report.mjs OK.
  • --dry-run --fixture (4 drift issues: qwen-code@0.9.0, qwen-code@0.8.0, a qwen-code docs issue, pi@0.9.0):
    • qwen-code success on 0.9.0 (Qwen Code): search -> comment on #1410 -> PATCH closed; docs issue, 0.8.0 and pi untouched.
    • qwen-code failure on v0.9.0 with the fixture's last verdict already failed for 0.9.0: comments read, not repeated.
    • qwen-code failure on 0.8.0: comment-only on #1411, no close.
    • sdk-langchain failure with --tested langchain-core=1.6.4,langgraph=: sweeps langchain-core@1.6.4 only (langgraph skipped, no version).

Not merged; the first scheduled run after merge is the live check.

@davidcrowe
davidcrowe merged commit 3afb007 into main Oct 8, 2026
2 checks passed
@davidcrowe
davidcrowe deleted the feat/canary-sweeps-drift branch October 8, 2026 02:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant