Skip to content

dev/main CI red: 6 deterministic coding-agent test failures shipped in v0.18.8 #6508

Description

@Yeachan-Heo

v0.18.8 (91fe978) was released by owner decision while these tests failed. The failures also occur on dev at 7acc11d (Dev CI run 37742470434) and on the main-push CI run 37743688666. They reproduce locally on the release commit every run:

  • test/agent-session-message-pipeline.test.ts: holds prompt settlement until worker integration is durable
  • test/async/job-manager-resume-queue.test.ts: a fenced queued owner remains queued while a later foreign owner resumes
  • test/harness-control-plane/session-lease.test.ts: heals an aged scrubbed removal transition instead of wedging the lease lock. Same lock-transition area as [Linux] Orphaned sessions/index.jsonl.lock.removing wedges the index lock; every broker startup dies on the 20s watchdog (1h outage) #6471.
  • test/eval/python-lifecycle.redteam.test.ts: settles an in-flight cell during shutdown without leaked abort listeners; bounds failed readiness cleanup before the fixture deadline; treats clean exit code 0 as confirmed and starts a fresh session instead of reinserting

These fail in CI only and did not reproduce locally: accepted-control zero-execution bound (#4668), with 2 cases; ACP /progress builtin prompt admission; and release-backmerge.test.ts diverged-dev merge, which passes locally.

Not yet determined: whether each failure is a product regression or a stale test expectation. Each one needs root-causing before the next release.

Separately, the main-push CI fails 3 compiled-binary smoke tests with pi_natives.linux-x64-baseline.node is present but has no trusted digest. That failure was already there at v0.18.7 (f2bba35) and does not affect tag builds.

Activity

  1. Yeachan-Heo commented on Oct 8, 2026

    @Yeachan-Heo
    OwnerAuthor

    Root-caused the issue cases against the reported dev CI and reproduced the listed deterministic failures locally on the issue baseline. PR #6512 carries the fixes and test-contract corrections.

    • AgentSession worker-integration: the test used worker-request start plus Bun.sleep(0) as a barrier for the second agent_end; those events have no such ordering guarantee. The test now awaits the actual event while retaining the worker-integration settlement assertions. Classification: stale test synchronization.
    • Async queued resume: shutdown proof captured A's previous job identity, while queued cancellation produced terminal evidence for queued:A:<seq>. The fix binds proof, queue draining, message updates, and cancellation to the captured owner and sequence. Classification: product regression.
    • Session lease: the failing fixture left an ownerless zero-byte info file. Age alone does not prove a scrubbed transition; current recovery requires the exact dead-owner record or an aged empty root. Tests now cover verified dead-owner recovery, live-owner refusal, and fail-closed ownerless residue. Classification: stale test fixture, not a reason to weaken lock ownership checks.
    • Python lifecycle: beginPythonRequest gives PythonKernel.execute an AbortSignal.any composite, not the original controller signal; shutdown confirmation is based on a non-null exit code, including 0. The red-team fakes/assertions now observe/model those contracts and verify unconfirmed cleanup remains retained and retryable. Classification: stale test expectations; no executor change.
    • Accepted-control bug: SDK accepts prompts while execution remains permanently zero-activity #4668: one case now waits until the tool-set seam observes drain rather than treating a test-set mutation as a wake signal. The other exposed a product race: without a tool-set seam, deadline ownership observation was not captured, allowing an agent_end to terminalize from a provider diagnostic without exact settlement evidence. The runtime now captures lifecycle ownership independently and still fails closed without proof.
    • ACP /progress: the admission test started before the asynchronous bootstrap update; it now waits for available_commands_update, matching the passing real-path test. Admission/cancel/error assertions remain. Classification: stale test readiness setup.
    • Diverged-dev release-backmerge.test.ts: the old direct-merge expectation conflicts with protected-dev PR policy. PR chore(release): backmerge v0.18.8 into dev #6506 (merge commit 1a6aa6721a4bb3dd6c2ce2003c3abe465188ff72) updated the backmerge flow and its test; it now passes locally and is not duplicated in fix(coding-agent): resolve v0.18.8 CI lifecycle regressions #6512.

    After rebasing onto dev 60d0b94ca28b860e504d6068934b9541e60ec915, all 10 focused files passed locally, including the affected issue tests and the extra CI-only task-artifact ownership test. bun --cwd=packages/coding-agent run check passed with four existing warnings; no warnings were suppressed. PR exact-head CI is running.

    —
    [repo owner's gaebal-gajae (clawdbot) 🦞]

  2. probepark commented on Oct 8, 2026

    @probepark
    Collaborator

    Classification: B — owner-directed remediation is already underway.
    Type: bug
    Priority: P1 — retain the next-release blocker until the affected failures are resolved.

    Evidence:

    • Issue #6508 reports failures shipped in v0.18.8 by owner decision, with both deterministic local failures and additional CI-only failures.
    • Owner Yeachan-Heo’s update reports root causes and fixes/test-contract corrections in PR #6512: product regressions in queued-resume ownership and accepted-control settlement, plus stale synchronization, fixtures, or expectations in the other cases. The owner reports that all 10 focused files and the coding-agent check passed locally, with four existing warnings; exact-head CI was still running at the time of that comment. These are the owner’s reported results, not independent verification.
    • The owner reports that PR #6506 already addressed the protected-dev backmerge contract; do not duplicate that fix.

    Next action: Continue the owner’s existing remediation through PR #6512 rather than opening duplicate implementation work. Record its final exact-head CI result here and account for each originally reported failure before closing this issue or clearing the next-release blocker. Track the pre-existing compiled-binary trusted-digest failures separately; the issue reports that they predate v0.18.8 and do not affect tag builds.

    The owner’s decision to release v0.18.8 remains unchanged. This triage creates no card and does not authorize a release or lift any user freeze.

  3. added 6 commits that reference this issue on Oct 9, 2026
    4e0c334
    6046f2b
    7c0a7ed
    39d1450
    cac68d8
    8368c7b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High prioritybugSomething isn't workingrelease-blockerBlocks the next release

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions