Repository navigation
harness canary: the scheduled run resolves matching drift issues (no dispatch token) - #52
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Founder decision: no
CANARY_DISPATCH_TOKEN. Instead of the drift scout triggering the canary, the scheduled canary (every 6h,@latest) finds and resolves the scout's release issues itself.What changes
scripts/canary-report.mjsdriftissues indavidcrowe/gatewaystack-connect, keeps those whose body carries<!-- drift-scout:key=release:<entryId>@<version> -->for an entry mapped to this leg (LEG_ENTRIESmirrorscanaryLegin the scout'sregistry.ts: 11 harnesses by name,crewai->sdk-crewai,langchain-core+langgraph->sdk-langchain,anthropic-sdk-typescript->sdk-governance-anthropic) and whose normalised version (samenormalizeVersionrule as the scout'sissue.ts) equals the version this run tested.Canary <leg> on <version>: ✅ passed — closing.and closes (state_reason: completed).Canary <leg> on <version>: ❌ failed at <step>: <canary issue / run link>once per failure; a hidden<!-- harness-canary:verdict leg=.. version=.. result=.. -->marker lets a later run see that its last verdict on that issue already says failed for the same version and not repeat it.release:markers) are never touched. Best effort: a sweep error is logged, never fails the reporter.--drift-issuepairing path (harness canary: post the verdict on the drift issue that dispatched the run #51) unchanged; the sweep skips that issue so it is not commented twice.--tested entryId=version,...for SDK legs (their--versionis the ACP package's, not the upstream's); an SDK entry with no pair is skipped. New--fixture <json>for--dry-runpreviews..github/workflows/harness-canary.ymlinstall-packagenow also records the upstream versions it installed (langchain-core,langgraph,crewai,@anthropic-ai/sdk) as anupstreamoutput, passed to the reporter as--tested.repository_dispatchkept for hand-triggered runs.docs/harness-canary.md: "Drift pairing" replaced by "Drift sweep": the scheduled canary resolves matching drift issues within ~6h; no extra token.CANARY_DISPATCH_TOKENinstructions removed.Validation
npx --yes js-yaml .github/workflows/harness-canary.ymlOK;node --check scripts/canary-report.mjsOK.--dry-run --fixture(4 drift issues: qwen-code@0.9.0, qwen-code@0.8.0, a qwen-code docs issue, pi@0.9.0):qwen-codesuccess on0.9.0 (Qwen Code): search -> comment on #1410 -> PATCH closed; docs issue, 0.8.0 and pi untouched.qwen-codefailure onv0.9.0with the fixture's last verdict alreadyfailedfor 0.9.0: comments read, not repeated.qwen-codefailure on0.8.0: comment-only on #1411, no close.sdk-langchainfailure with--tested langchain-core=1.6.4,langgraph=: sweepslangchain-core@1.6.4only (langgraph skipped, no version).Not merged; the first scheduled run after merge is the live check.