Skip to content

Track upstream agent CLI versions - #1637

Open
a5c-ai[bot] wants to merge 1 commit into
stagingfrom
agent-versions/daily-2026-08-05
Open

Track upstream agent CLI versions#1637
a5c-ai[bot] wants to merge 1 commit into
stagingfrom
agent-versions/daily-2026-08-05

Conversation

@a5c-ai

@a5c-ai a5c-ai Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Updates Atlas AgentVersion records from the daily upstream host agent release check.

Artifacts:

  • artifacts/agent-version-tracker/upstream-targets-and-latest.json
  • artifacts/agent-version-tracker/summary.json

Verification:

  • git diff --check
  • npm run build --workspace=@a5c-ai/atlas

Note: npm run verify:metadata was attempted after PR creation but the local workspace has unrelated dirty plugin-marketplace changes outside this PR, causing the check to fail before evaluating these graph changes.

@a5c-ai a5c-ai Bot mentioned this pull request Aug 5, 2026
3 tasks
@a5c-ai

a5c-ai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Adversarial review result: request changes.

I reviewed the PR metadata, full diff, all changed files from origin/pr/1637, existing comments/reviews, CI state, and a clean temp worktree at /tmp/pr-1637-review-01KZA8.

Blockers

  1. Docs QA is failing on this PR.

    gh pr checks 1637 reports Docs QA as failed. The failed job log shows docs:freshness detecting stale generated documentation:

    • docs/adapters/reference/04-agent-events.md: generated doc is stale (56d > 45d)
    • docs/harness-features-backlog/gaps/prompt-engineering/GAP-PROMPT-011.md: generated doc is stale (56d > 45d)

    This appears unrelated to the graph files changed in this PR, but it is still a red CI gate. Please refresh/fix the stale generated docs or rerun after the base branch fixes the docs freshness failure. I do not think we should merge while the required review signal is red.

Major Findings

  1. The generated latest-version artifacts are already stale against current npm latest values.

    The PR records npm latest values in artifacts/agent-version-tracker/upstream-targets-and-latest.json starting at line 132, and mirrors those values in artifacts/agent-version-tracker/summary.json. Fresh npm checks during review returned newer latest dist-tags for several packages:

    • @ampcode/cli: PR has 0.0.1785906080-gd4ffa5; current npm latest is 0.0.1785975399-gb63fe6
    • @anthropic-ai/claude-agent-sdk: PR has 0.3.222; current npm latest is 0.3.223
    • @anthropic-ai/claude-code: PR has 2.1.222; current npm latest is 2.1.223
    • @factory/cli: PR has 0.188.0; current npm latest is 0.189.0
    • @openai/codex: PR has 0.146.0; current npm latest is 0.146.1
    • @qwen-code/qwen-code: PR has 0.21.5; current npm latest is 0.21.6
    • opencode-ai: PR has 1.18.13; current npm latest is 1.18.14

    Because this PR is specifically a current upstream version tracker output, please rerun the tracker on the PR branch before merge so the generated artifacts and AgentVersion YAML include the versions currently published.

QA / Verification

Passed locally or in existing CI:

  • git diff --check origin/staging...HEAD
  • npm run verify:metadata
  • Existing PR CI: Lint, Tests, Package, Workspace Coverage, and Observer Dashboard

Could not fully reproduce locally:

  • npm run build --workspace=@a5c-ai/atlas in a clean temp worktree failed before evaluating PR code because dependencies were not installed there (tsx: not found). Existing CI Lint, Tests, Package passed, so this is a local reproduction limitation, not a proven code failure.

Additional QA dispatch:

  • Dispatched qa-dispatch.yml run 31061121435 for branch agent-versions/daily-2026-08-05 and PR 1637.
  • After roughly five minutes it was still in progress at job 92489127792, step Run a5c-ai/babysitter/packages/adapters/triggers@staging, with logs unavailable until completion.
  • Treated as inconclusive for this review decision.

Risk Assessment

Risk level: risk:medium

  • Data freshness risk: upstream npm packages release frequently, and this PR is already behind several latest dist-tags.

    • Pre-merge mitigation: rerun the tracker and verify npm latest values.
    • Deploy-time mitigation: no special rollout needed because this is catalog data.
    • Post-merge mitigation: schedule the next daily tracker run and compare for omitted versions.
  • CI gate risk: Docs QA is failing independently of these graph files.

    • Pre-merge mitigation: require Docs QA to pass, or prove it is a base-branch-only failure and rerun after repair.
    • Deploy-time mitigation: do not merge with failing required checks.
    • Post-merge mitigation: monitor docs freshness so unrelated stale docs do not block future graph updates.

@a5c-ai

a5c-ai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Run: https://github.com/a5c-ai/babysitter/actions/runs/31061259641

Result: pending / timed out waiting for final CI status. The live-stack workflow was dispatched for the Atlas graph/catalog metadata update and was still in_progress when the QA process reached its 20-minute wait limit.

Job Status Conclusion
Build All in_progress pending
Compute Matrix completed success

Focused matrix tested:

[
  {"agent":"codex","model":"google-gemini31","mode":"ni","install":"vanilla","live":true},
  {"agent":"claude","model":"foundry-gpt55","mode":"ni","install":"vanilla","live":true},
  {"agent":"codex","model":"google-gemini31","mode":"interactive","install":"bp","live":true,"process_mode":"predefined"},
  {"agent":"claude","model":"foundry-gpt55","mode":"interactive","install":"bp","live":true,"process_mode":"predefined"},
  {"agent":"codex","model":"google-gemini31","mode":"bridged-hooks","install":"bp","live":true,"process_mode":"predefined"}
]

Verdict: not passed yet. No live-stack job failed at the time of review, but final QA cannot be marked passing until the run completes.

@a5c-ai

a5c-ai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing yet.

I dispatched live-stack QA for the Atlas graph release-record update on agent-versions/daily-2026-08-05:

https://github.com/a5c-ai/babysitter/actions/runs/31061253860

Focused matrix:

Agent Model Mode Install Process mode
codex foundry-gpt55 interactive bp predefined
codex foundry-gpt55 interactive bp create
claude foundry-gpt55 interactive bp predefined
claude foundry-gpt55 ni vanilla -

Current live-stack status after the QA polling window:

Job Status Result
Compute Matrix completed success
Build All in_progress pending

Verdict: no pass verdict yet. The selected live-stack matrix has not run because Build All is still in progress.

Current PR checks observed during QA:

Check State
Docs QA FAILURE
Lint, Tests, Package SUCCESS
Workspace Coverage SUCCESS
Observer Dashboard SUCCESS

Scope note: PR #1637 changes Atlas AgentVersion graph records, catalog-meta evidence sources, and tracker artifacts. The matrix was selected to exercise BP/catalog-consuming paths plus one raw Claude adapter sanity check, rather than a broad transport/provider sweep.

@a5c-ai

a5c-ai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: not passed — workflow_dispatch run 31061330972 remained queued for the full 20-minute QA polling window, so no live-stack job results were produced.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31061330972
Branch/ref tested: agent-versions/daily-2026-08-05

Matrix requested

Agent Model Mode Install Process mode Live
codex google-gemini31 ni vanilla predefined true
claude foundry-gpt55 ni vanilla predefined true
hermes foundry-gpt55 ni vanilla predefined true
codex google-gemini31 interactive bp predefined true
claude foundry-gpt55 bridged-hooks bp predefined true
hermes foundry-gpt55 interactive bp create true

Result table

Job Result
workflow_dispatch run queued past 20 minute QA timeout timed_out_queued

Overall verdict: QA incomplete / not passing because the dispatched live-stack workflow did not start within the process timeout.

@a5c-ai

a5c-ai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: blocked / no verdict. The adversarial live-stack QA workflow was dispatched, but GitHub Actions run 31061295090 stayed queued through the 20-minute polling window, so no end-to-end pass/fail verdict is available yet.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31061295090

Current job status

Job Result
Compute Matrix success
Build All pending / no conclusion

Tested matrix

Agent Model Mode Install Process mode
codex foundry-gpt55 ni vanilla predefined
claude anthropic-sonnet46 ni vanilla predefined
gemini google-gemini31 bridged-interactive vanilla predefined
codex google-gemini31 interactive bp predefined
claude foundry-gpt55 interactive bp create
pi foundry-gpt55 bridged-hooks bp create

Rationale: PR changes Atlas graph/catalog agent-version data and tracker artifacts, so this focused adversarial matrix covers graph-consuming adapter/plugin paths across multiple agents, providers, interaction modes, and BP predefined/create flows without running the full cross-product.

@a5c-ai

a5c-ai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: inconclusive / not passed. The live-stack workflow was dispatched successfully, but it was still queued after the 20-minute QA polling window, so no job-level pass/fail results were available.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31061304452

Matrix tested

Agent Model Mode Install Live
claude anthropic-sonnet46 ni vanilla true
codex google-gemini31 ni vanilla true
pi foundry-gpt55 ni vanilla true
gemini google-gemini31 ni vanilla true
copilot foundry-gpt55 ni vanilla true
hermes foundry-gpt55 ni vanilla true

Job results

Job Result
No jobs completed within polling window queued / timeout

Overall verdict: not passed yet. Re-check the Actions run after runner capacity picks it up.

@a5c-ai

a5c-ai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete. The adversarial QA workflow was dispatched but did not complete within the 20-minute polling window.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31061319160

Job Result
Build All pending/no conclusion at timeout
Compute Matrix success

Matrix tested:

[{"agent":"codex","model":"foundry-gpt55","mode":"ni","install":"vanilla","live":true},{"agent":"claude","model":"anthropic-sonnet46","mode":"ni","install":"vanilla","live":true},{"agent":"codex","model":"google-gemini31","mode":"interactive","install":"bp","live":true,"process_mode":"predefined"},{"agent":"claude","model":"foundry-gpt55","mode":"bridged-hooks","install":"bp","live":true,"process_mode":"create"}]

Verdict: not all passed yet; QA remains pending on the live-stack run completion.

@a5c-ai

a5c-ai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: not completed. The focused live-stack QA run was dispatched but remained queued for the 20-minute polling window, so no job conclusions were available.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31061347114

Focused matrix

Agent Model Mode Install Process mode Result
codex foundry-gpt55 interactive bp predefined not run (queued timeout)
claude foundry-gpt55 interactive bp create not run (queued timeout)
codex google-gemini31 ni vanilla predefined not run (queued timeout)
gemini google-gemini31 ni vanilla predefined not run (queued timeout)

Selection rationale: PR #1637 changes Atlas graph-only upstream AgentVersion and catalog evidence-source records plus generated tracker artifacts. The focused matrix targets BP predefined/create catalog consumers plus vanilla Codex/Gemini adapter checks for generated graph/index regressions without running the full cross-product.

@a5c-ai

a5c-ai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: blocked. The focused live-stack workflow was dispatched, but it remained queued through the 20-minute polling window, so no job conclusions were available.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31061349100

Job Result
No jobs started before timeout queued / no conclusion

Matrix tested:

[
  {"agent":"codex","model":"google-gemini31","mode":"interactive","install":"bp","live":true,"process_mode":"predefined"},
  {"agent":"codex","model":"google-gemini31","mode":"interactive","install":"bp","live":true,"process_mode":"create"},
  {"agent":"claude","model":"foundry-gpt55","mode":"interactive","install":"bp","live":true,"process_mode":"predefined"},
  {"agent":"claude","model":"foundry-gpt55","mode":"bridged-hooks","install":"bp","live":true,"process_mode":"create"}
]

Rationale: PR #1637 changes Atlas graph/catalog AgentVersion and evidence-source records plus generated tracker artifacts, so the matrix focuses on BP agent-catalog consumers across predefined/create modes and one bridged-hooks path rather than a full transport/provider sweep.

@a5c-ai

a5c-ai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the tracker run. I’m requesting changes because the PR no longer represents the current upstream latest versions for several agents.

Blocker

artifacts/agent-version-tracker/upstream-targets-and-latest.json:135 and the surrounding npm latest entries are stale. The PR was generated at artifacts/agent-version-tracker/summary.json:2 (2026-08-05T07:38:33.458Z), but current upstream package metadata has advanced for multiple tracked packages:

  • @ampcode/cli: PR records 0.0.1785906080-gd4ffa5; npm latest is 0.0.1785975399-gb63fe6 published 2026-08-06T00:24:26.100Z.
  • @anthropic-ai/claude-agent-sdk: PR records 0.3.222; npm latest is 0.3.223 published 2026-08-05T22:50:10.676Z.
  • @anthropic-ai/claude-code: PR records 2.1.222; npm latest is 2.1.223 published 2026-08-05T22:51:13.206Z.
  • @factory/cli: PR records 0.188.0; npm latest is 0.189.0 published 2026-08-06T00:19:27.845Z.
  • @openai/codex: PR records 0.146.0; npm latest is 0.146.1 published 2026-08-05T16:00:31.441Z.
  • @qwen-code/qwen-code: PR records 0.21.5; npm latest is 0.21.6 published 2026-08-05T15:56:13.739Z.
  • opencode-ai: PR records 1.18.13; npm latest is 1.18.14 published 2026-08-05T20:56:10.249Z.

Those stale values are also propagated into packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml and packages/atlas/graph/catalog-meta/evidence-sources/upstream-current-2026-08-05.yaml, so merging this would publish already-outdated “latest/current” graph records.

Please re-run the daily tracker against current upstream metadata, refresh the artifacts and AgentVersion/EvidenceSource records, then re-run the Atlas build and metadata checks.

Verification

I checked the PR in an isolated worktree:

  • git diff --check origin/staging..origin/pr-1637-review: passed.
  • npm run verify:metadata: passed in the PR worktree. The current checkout still has the unrelated .agents/plugins/marketplace.json metadata issue noted in the PR body.
  • npm run build --workspace=@a5c-ai/atlas: passed after installing dependencies in the isolated PR worktree.
  • Atlas SDK exact lookups for the new version records resolve.
  • Direct version_of and sourced_from edges touching the new records have no missing endpoints.

QA dispatch was inconclusive/not passed: qa-dispatch.yml completed, but the downstream live-stack run remained queued through its polling window. The QA process posted details in #1637 (comment).

Risk Assessment

Risk level: risk:medium.

Risk: Atlas consumers may read stale latest/current agent version data immediately after merge.
Mitigation: refresh the tracker output immediately before merge and verify the current npm/GitHub release metadata again.

Risk: the Atlas build can exit 0 while reporting stats.parseErrors = 1 and library bridge quality failures. This appears broader than this PR, but it weakens confidence in graph-only changes.
Mitigation: track a follow-up to make strict graph validation fail on parse errors and required bridge-quality failures, or document which failures are intentionally non-blocking.

@a5c-ai

a5c-ai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Request changes

I found one blocker that needs to be fixed before this can merge.

Blocker: published "latest" version records are stale before merge

The PR records npm latest values captured on 2026-08-05, but live package metadata checked during review on 2026-08-06 has already advanced for multiple packages:

Package PR records Current npm latest
@ampcode/cli 0.0.1785906080-gd4ffa5 0.0.1785975399-gb63fe6
@anthropic-ai/claude-agent-sdk 0.3.222 0.3.223
@anthropic-ai/claude-code 2.1.222 2.1.223
@factory/cli 0.188.0 0.189.0
@openai/codex 0.146.0 0.146.1
@qwen-code/qwen-code 0.21.5 0.21.6
opencode-ai 1.18.13 1.18.14

Affected PR files include:

  • artifacts/agent-version-tracker/upstream-targets-and-latest.json, where these values are recorded in npmLatest.
  • artifacts/agent-version-tracker/summary.json:13, :21, :29, :61, :101, :117, :125 and the matching newVersions / issue sections later in the file.
  • packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:7, :74, :114, :144, :286, :459, :533.
  • packages/atlas/graph/catalog-meta/evidence-sources/upstream-current-2026-08-05.yaml:18 and :250, which claim stale npm metadata as the latest dist-tag for at least Amp and Droid.

This is a correctness blocker because the PR's purpose is to track current upstream agent CLI versions. Merging it now would publish outdated versions as the current catalog state.

Fix: rerun/regenerate the tracker against current upstream metadata, update the AgentVersion and EvidenceSource records, and create/update tracking issues for any newly observed versions before merge.

Major: time-sensitive evidence wording is too absolute

packages/atlas/graph/catalog-meta/evidence-sources/upstream-current-2026-08-05.yaml:18 and :250 say npm metadata "reports" those versions as the latest dist-tag. That wording becomes false as soon as the dist-tag moves. If the record is meant to be historical evidence, phrase it as an observed-at/captured-at fact instead of a global latest claim.

QA

Local checks:

  • npm run verify:metadata passed in a clean detached PR worktree.
  • npm run build --workspace=@a5c-ai/atlas exited 0 after installing dependencies in the detached worktree. The build prints existing library-bridge semantic edge warnings, but they did not fail the command.

Live-stack QA dispatch:

  • qa-dispatch.yml run 31061193330 completed successfully as a dispatcher.
  • The inner live-stack QA was blocked/not completed: runs 31061347114 and 31061349100 remained queued for the 20-minute polling window, so no live-stack job conclusions were available.
  • QA comments were posted on this PR with the blocked result.

Risk Assessment

Risk level: risk:medium

  • Risk: Atlas consumers ingest outdated upstream version records as current for multiple agents.
    Mitigation: refresh the tracker data before merge and verify current npm/GitHub release metadata immediately before approval.

  • Risk: this stale-output pattern recurs for fast-moving packages.
    Mitigation: add a lightweight freshness guard for package-backed latest records, or at least add a pre-merge check that compares recorded npm latest values against npm view <package>@latest version.

@a5c-ai

a5c-ai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Blocking review: QA incomplete

The code/data review did not find blockers or major issues in the PR contents themselves. I verified the added Atlas records and evidence files in an isolated PR worktree:

  • git diff --check origin/staging...HEAD passed.
  • npm run build --workspace=@a5c-ai/atlas passed after installing dependencies in the isolated worktree.
  • npm run verify:metadata passed.
  • The new AgentVersion and EvidenceSource YAML files parse cleanly and have no duplicate IDs.

However, the adversarial review process has a strict gate: do not approve while QA is failed or inconclusive. The dispatched QA wrapper completed, but the nested live-stack run is still in progress and has not produced an all-passing terminal result yet.

QA evidence:

Please wait for the nested live-stack run to complete and re-request review once it has a passing terminal conclusion.

Risk Assessment

Risk level: risk:low

  • Risk: additive Atlas graph/evidence data could be malformed or introduce duplicate IDs. Mitigation: YAML parsing, duplicate-ID checks, Atlas build, metadata verification, and diff whitespace checks passed.
  • Risk: the snapshot is dated 2026-08-05 and some npm dist-tags had already advanced by review time on 2026-08-06. Mitigation: treat this as a dated snapshot; the next scheduled daily tracker run should capture subsequent releases.
  • Risk: live-stack regressions remain unknown until QA completes. Mitigation: keep the PR unapproved until the nested live-stack run finishes successfully.

Note: GitHub would not allow this actor to submit a formal request-changes review on its own PR, so this comment carries the blocking decision.

@a5c-ai

a5c-ai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passed. The focused adversarial live-stack workflow was dispatched, but it did not reach a terminal conclusion within the 20-minute QA polling window.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31138625722
Branch/ref tested: agent-versions/daily-2026-08-05

Job results at timeout

Job Status Result
Build All in_progress pending
Compute Matrix completed success

Focused matrix

Agent Model Mode Install Process mode
codex google-gemini31 interactive bp predefined
claude foundry-gpt55 interactive bp create
codex google-gemini31 bridged-hooks bp predefined
claude anthropic-sonnet46 ni vanilla -
hermes foundry-gpt55 ni vanilla -

Verdict: not passing yet. No live-stack job failed during the polling window, but final QA cannot be marked passing until the workflow completes successfully.

@a5c-ai

a5c-ai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing yet. The focused live-stack workflow was dispatched for the Atlas AgentVersion graph/catalog metadata update, but it did not reach a terminal conclusion within the 20-minute QA polling window.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31138660127
Branch/ref tested: agent-versions/daily-2026-08-05

Job results at timeout

Job Status Conclusion
Build All in_progress pending
Compute Matrix completed success

Focused matrix

Agent Model Mode Install Process mode Live
codex google-gemini31 interactive bp predefined true
claude foundry-gpt55 interactive bp create true
codex google-gemini31 bridged-hooks bp predefined true
claude foundry-gpt55 ni vanilla predefined true
gemini google-gemini31 ni vanilla predefined true

Selection rationale: PR #1637 changes Atlas AgentVersion graph records, catalog-meta evidence sources, and generated tracker artifacts, so this matrix focuses on BP catalog-consuming flows across predefined/create plus a bridged-hooks plugin path and vanilla Claude/Gemini adapter sanity checks.

Overall verdict: not passed at QA cutoff. No live-stack job failed during the polling window, but final QA cannot be marked passing until the run completes successfully.

@a5c-ai

a5c-ai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing yet. The adversarial live-stack workflow was dispatched successfully, but it did not complete within the 20-minute QA polling window.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31138670464
Branch/ref tested: agent-versions/daily-2026-08-05

Current job status

Job Status Result
Build All in_progress pending
Compute Matrix completed success

Focused matrix

Agent Model Mode Install Process mode Live
codex google-gemini31 interactive bp predefined true
codex google-gemini31 bridged-hooks bp create true
claude foundry-gpt55 interactive bp predefined true
claude foundry-gpt55 interactive bp create true
gemini google-gemini31 ni vanilla - true
hermes foundry-gpt55 ni vanilla - true

Rationale: PR #1637 changes Atlas AgentVersion graph records, catalog-meta evidence sources, and generated tracker artifacts. This matrix focuses on BP/catalog-consuming paths across predefined/create modes, one bridged-hooks path, and vanilla Gemini/Hermes adapter sanity checks without running a full unrelated cross-product.

Overall verdict: not passed yet. No live-stack job failed during the polling window, but final QA cannot be marked passing until the run completes with successful job conclusions.

@a5c-ai

a5c-ai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing yet.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31138659034

The adversarial live-stack QA workflow was dispatched for agent-versions/daily-2026-08-05, but it did not reach a terminal result within the 20-minute QA polling window.

Job Status Result
Build All in_progress pending
Compute Matrix completed success

Focused matrix tested:

Agent Model Mode Install Process mode
codex google-gemini31 interactive bp predefined
claude foundry-gpt55 interactive bp create
codex google-gemini31 bridged-hooks bp predefined
hermes foundry-gpt55 ni vanilla -
claude anthropic-sonnet46 ni vanilla -

Verdict: QA is not passing yet. No live-stack job failed during the polling window, but final QA cannot be marked passing until the run completes successfully.

@a5c-ai

a5c-ai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing yet. The adversarial live-stack workflow was dispatched successfully, but it did not complete within the 20-minute QA polling window.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31138667125

Job Status Conclusion
Build All in_progress pending
Compute Matrix completed success

Focused matrix tested:

[
  {"agent":"codex","model":"google-gemini31","mode":"interactive","install":"bp","live":true,"process_mode":"predefined"},
  {"agent":"codex","model":"foundry-gpt55","mode":"bridged-hooks","install":"bp","live":true,"process_mode":"predefined"},
  {"agent":"claude","model":"foundry-gpt55","mode":"interactive","install":"bp","live":true,"process_mode":"create"},
  {"agent":"claude","model":"anthropic-sonnet46","mode":"ni","install":"vanilla","live":true},
  {"agent":"hermes","model":"foundry-gpt55","mode":"ni","install":"vanilla","live":true}
]

Rationale: PR #1637 changes Atlas AgentVersion graph/catalog records and tracker artifacts, so this matrix focuses on BP catalog-consuming paths across predefined/create and bridged-hooks flows, with vanilla Claude/Hermes checks to catch adapter/catalog regressions without running the full cross-product.

Overall verdict: not passed yet. No live-stack job failed during the polling window, but final QA cannot be marked passing until the run reaches a terminal successful conclusion.

@a5c-ai

a5c-ai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: not passed / timed out waiting for final live-stack status.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31138672704
Branch/ref tested: agent-versions/daily-2026-08-05

The workflow was queued for most of the 20-minute QA polling window and had only reached in_progress at the final poll. No live-stack execution jobs produced terminal conclusions before timeout.

Job status at timeout

Job Status Conclusion
Compute Matrix completed success
Build All in_progress pending

Focused matrix

Agent Model Mode Install Process mode Live
codex google-gemini31 ni vanilla predefined true
claude foundry-gpt55 ni vanilla predefined true
gemini google-gemini31 ni vanilla predefined true
codex google-gemini31 interactive bp predefined true
claude foundry-gpt55 interactive bp create true
hermes foundry-gpt55 bridged-hooks bp predefined true

Rationale: PR #1637 changes Atlas AgentVersion graph records, catalog evidence sources, and generated tracker artifacts, so this focused matrix covers vanilla graph/catalog metadata consumers plus BP catalog-consuming predefined/create paths and one bridged-hooks BP path.

Overall verdict: QA incomplete / not passing because the dispatched live-stack workflow did not finish within the QA timeout window.

@a5c-ai

a5c-ai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing. The adversarial live-stack workflow was dispatched successfully, but Build All remained queued for the full 20-minute QA polling window, so no live-stack scenario job conclusions were produced.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31138694373
Branch/ref tested: agent-versions/daily-2026-08-05

Job results

Job Status Result
Compute Matrix completed success
Build All queued pending / timed out waiting for runner

Matrix requested

Agent Model Mode Install Process mode Live
codex google-gemini31 interactive bp predefined true
claude foundry-gpt55 interactive bp create true
codex google-gemini31 bridged-hooks bp resume true
hermes foundry-gpt55 ni vanilla predefined true
gemini google-gemini31 ni vanilla predefined true
claude anthropic-sonnet46 ni vanilla predefined true

Selection rationale: PR #1637 changes Atlas AgentVersion graph data, catalog-meta evidence records, and tracker artifacts rather than transport code. This focused adversarial matrix targets BP/catalog-consuming paths across predefined, create, and resume process modes, includes bridged-hooks for hook-mediated BP behavior, and adds representative vanilla adapter sanity checks for Hermes, Gemini, and Claude/Anthropic.

Overall verdict: not passed yet. The dispatch succeeded, but the live-stack tests did not execute within the QA process timeout.

@a5c-ai

a5c-ai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing yet. The focused adversarial live-stack workflow was dispatched, but it did not reach a terminal all-passing result within the 20-minute QA polling window.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31138694225
Branch/ref tested: agent-versions/daily-2026-08-05

Current job status at timeout

Job Status Result
Build All in_progress pending
Compute Matrix completed success

Focused matrix

Agent Model Mode Install Process mode
codex google-gemini31 ni vanilla -
claude foundry-gpt55 ni vanilla -
codex google-gemini31 interactive bp predefined
claude foundry-gpt55 bridged-hooks bp create

Verdict: not passed because the live-stack run had no terminal conclusion before the QA timeout. Re-check the Actions run after Build All completes.

@a5c-ai

a5c-ai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Adversarial review result: request changes.

I reviewed the PR metadata, full diff, changed files from origin/pr-1637-review, existing comments/reviews, CI state, generated Atlas graph records, tracker artifacts, and live npm metadata on 2026-08-07.

Blockers

  1. Generated current/latest records are stale against live upstream package metadata.

    The PR records package latest values captured on 2026-08-05 in artifacts/agent-version-tracker/upstream-targets-and-latest.json:134 and mirrors them through artifacts/agent-version-tracker/summary.json:13, packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:3, and related AgentVersion/EvidenceSource records.

    Live npm metadata checked during this review has already advanced for multiple package-backed agents:

    Package PR records Current npm latest
    @ampcode/cli 0.0.1785906080-gd4ffa5 0.0.1786064749-gf2437d
    @anthropic-ai/claude-agent-sdk 0.3.222 0.3.223
    @anthropic-ai/claude-code 2.1.222 2.1.223
    @factory/cli 0.188.0 0.189.0
    @openai/codex 0.146.0 0.146.1
    @qwen-code/qwen-code 0.21.5 0.21.7
    opencode-ai 1.18.13 1.18.14
    @google/gemini-cli 0.53.1 0.54.0
    @oh-my-pi/pi-coding-agent 17.2.9 17.2.10
    @earendil-works/pi-coding-agent 0.83.0 0.84.0

    Because this PR's purpose is publishing upstream current agent-version records, merging known-stale current records would make Atlas catalog data incorrect immediately. Please rerun/regenerate the tracker immediately before merge, refresh the artifacts plus AgentVersion/EvidenceSource records, then rerun metadata and Atlas build verification.

  2. Required CI signal is red: Docs QA failed.

    The PR check rollup shows Docs QA completed with FAILURE; Lint, Tests, Package, Workspace Coverage, and Observer Dashboard passed. Even if the docs freshness failure is unrelated to these graph files, this review process should not approve or merge while a required review signal is red. Please fix/refresh the stale docs on the branch or base and rerun CI, or document a repository-approved exception before merge.

Major Finding

  1. Time-sensitive package evidence wording is too absolute for historical records.

    packages/atlas/graph/catalog-meta/evidence-sources/upstream-current-2026-08-05.yaml:18 and :250 say npm metadata reports a version as the latest dist-tag. That wording becomes false as soon as the dist-tag moves, even if it was true at observedAt. Please phrase these as observed-at facts, for example: "At observedAt, npm metadata reported ...".

QA

I dispatched the adversarial QA workflow for agent-versions/daily-2026-08-05.

  • Dispatcher run: 31138464043 completed successfully.
  • Downstream live-stack run: 31138659034.
  • Result: incomplete / not passing yet. After the downstream 20-minute polling window, Compute Matrix had passed, but Build All was still in_progress.
  • QA comment posted: Track upstream agent CLI versions #1637 (comment)

Under the review process rules, incomplete QA is not a passing verdict.

Risk Assessment

Risk level: risk:medium.

  • Risk: Atlas/catalog consumers ingest stale upstream current agent-version records immediately after merge.

    • Pre-merge mitigation: rerun the tracker and compare every package-backed record against live npm metadata.
    • Deploy-time mitigation: do not merge until generated data is fresh.
    • Post-merge mitigation: schedule the next tracker run and monitor for omitted fast-follow releases.
  • Risk: repository quality gates are weakened if this graph-data PR merges while Docs QA is red.

    • Pre-merge mitigation: require Docs QA to pass or document an approved base-branch-only exception.
    • Deploy-time mitigation: keep merge blocked while required checks are failing.
    • Post-merge mitigation: track docs freshness separately so unrelated stale docs do not repeatedly block graph updates.
  • Risk: historical evidence records become misleading as moving dist-tags advance.

    • Pre-merge mitigation: change evidence wording to observed-at language.
    • Post-merge mitigation: add review guidance or a lint/check for time-sensitive evidence phrases.

@a5c-ai

a5c-ai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Requesting changes. I found two blockers and one major issue.

Blockers

  1. The generated latest-version records are stale against current upstream package metadata.

    artifacts/agent-version-tracker/upstream-targets-and-latest.json:132 records package-backed npmLatest values that no longer match current npm latest values checked during this review on 2026-08-07:

    Package PR records Current npm latest
    @ampcode/cli 0.0.1785906080-gd4ffa5 0.0.1786064749-gf2437d
    @anthropic-ai/claude-agent-sdk 0.3.222 0.3.223
    @anthropic-ai/claude-code 2.1.222 2.1.223
    @factory/cli 0.188.0 0.189.0
    @openai/codex 0.146.0 0.146.1
    @qwen-code/qwen-code 0.21.5 0.21.7
    opencode-ai 1.18.13 1.18.14

    I also checked @google/gemini-cli and @earendil-works/pi-coding-agent; they are now 0.54.0 and 0.84.0, while artifacts/agent-version-tracker/summary.json:69 and :109 record 0.53.1 and 0.83.0.

    This PR publishes dated Atlas currentVersion graph records, for example packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:10, :77, :117, :147, :289, :462, and :536. Because the PR's purpose is current upstream version tracking, merging already-stale latest/current data would mislead Atlas/catalog consumers.

    Fix: rerun the tracker against current upstream metadata, regenerate the artifacts and AgentVersion/EvidenceSource records, and update/create any tracking issues for newly observed versions.

  2. Docs QA is failing on this PR.

    gh pr checks 1637 reports Docs QA as FAILURE. The failed job log shows docs:freshness found stale generated docs:

    • docs/adapters/reference/04-agent-events.md: 56d > 45d
    • docs/harness-features-backlog/gaps/prompt-engineering/GAP-PROMPT-011.md: 56d > 45d

    This may be unrelated to the graph files, but the PR is not in a mergeable quality state while a required review signal is red.

    Fix: refresh/repair the stale generated docs, or rebase/rerun after the base branch fixes the freshness failure, then require Docs QA to pass.

Major

packages/atlas/graph/catalog-meta/evidence-sources/upstream-current-2026-08-05.yaml:18 and :250 use present-tense latest wording: npm metadata "reports" the recorded package versions as the latest dist-tag. Those statements become false as soon as the dist-tag moves, and several have already moved.

Fix: phrase dated package metadata as observed-at evidence, for example "At observedAt, npm metadata reported ... as latest", rather than a global present-tense claim.

QA

I dispatched qa-dispatch.yml for agent-versions/daily-2026-08-05 and PR 1637.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31138487516

Result: inconclusive / not passed. After the polling window, job qa was still in_progress in step Run a5c-ai/babysitter/packages/adapters/triggers@staging; no live-stack pass/fail result was available.

Risk Assessment

Risk level: risk:medium

  • Risk: Atlas consumers ingest outdated upstream AgentVersion records as current for multiple agents.
    Mitigation: rerun/regenerate the tracker immediately before approval and verify package/GitHub release metadata again.

  • Risk: CI quality signal is red because Docs QA failed.
    Mitigation: require Docs QA to pass before approval or prove and repair a base-only freshness failure before review.

  • Risk: dated evidence records become misleading when they use present-tense latest wording.
    Mitigation: rewrite package metadata evidence as explicit observed-at snapshots.

@a5c-ai

a5c-ai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Request changes

I found blockers that need to be fixed before this can merge.

Blockers

  1. The generated current/latest records are stale against live upstream package metadata.

    The PR records npm latest values from 2026-08-05, but current npm metadata checked on 2026-08-07 has already advanced:

    Package PR records Current npm latest
    @ampcode/cli 0.0.1785906080-gd4ffa5 0.0.1786064749-gf2437d
    @anthropic-ai/claude-agent-sdk 0.3.222 0.3.223
    @anthropic-ai/claude-code 2.1.222 2.1.223
    @factory/cli 0.188.0 0.189.0
    @openai/codex 0.146.0 0.146.1
    @qwen-code/qwen-code 0.21.5 0.21.7
    opencode-ai 1.18.13 1.18.14

    Affected files include artifacts/agent-version-tracker/upstream-targets-and-latest.json:134, :138, :142, :146, :150, :158, and :166, plus the matching currentVersion AgentVersion records in packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:3, :70, :110, :140, :282, :455, and :529.

    Because this PR's purpose is to publish current upstream agent-version records, merging it now would knowingly publish stale current catalog data. Please rerun/regenerate the tracker against current upstream metadata immediately before merge, update the AgentVersion/EvidenceSource records and tracker artifacts, and ensure matching tracking issues exist for newly observed versions.

  2. Docs QA is failing.

    gh pr checks 1637 reports Docs QA as failed. The job log shows docs:freshness failed because these generated docs are stale:

    • docs/adapters/reference/04-agent-events.md: 56d > 45d
    • docs/harness-features-backlog/gaps/prompt-engineering/GAP-PROMPT-011.md: 56d > 45d

    Please refresh/fix the stale generated docs or rebase/rerun after the base branch fixes them. I would not merge with a red QA gate.

Major

The evidence wording is too absolute for time-sensitive npm metadata. packages/atlas/graph/catalog-meta/evidence-sources/upstream-current-2026-08-05.yaml:18 and :250 say npm metadata reports those versions as the latest dist-tag. That was only true at the recorded observation time and is already false. When regenerating, phrase this as observed-at evidence rather than a timeless latest claim.

QA

I dispatched qa-dispatch.yml for agent-versions/daily-2026-08-05 and PR 1637.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31138483725

After 25 one-minute polls, the dispatcher job was still in_progress in Run a5c-ai/babysitter/packages/adapters/triggers@staging, so no terminal QA pass/fail verdict or downstream live-stack conclusion was available. Under this review process, QA is inconclusive/not passed.

Risk Assessment

Risk level: risk:medium.

  • Risk: Atlas/catalog consumers ingest stale upstream version data as current for multiple agent CLIs.
    Mitigation: regenerate against current npm/GitHub metadata and rerun Atlas build plus metadata verification before merge.

  • Risk: the PR does not have a fully green quality signal because Docs QA is failing and live-stack QA did not reach a passing terminal result.
    Mitigation: require Docs QA to pass and rerun/check live-stack QA after runner capacity clears.

  • Risk: fast-moving package-backed latest records will keep becoming stale during review.
    Mitigation: add a follow-up guardrail that compares recorded npm latest values against npm view <package>@latest version before approving generated latest/current tracker PRs.

@a5c-ai

a5c-ai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Adversarial review result: request changes.

I reviewed the PR metadata, full diff, changed files from origin/pr/1637, existing comments, current PR checks, and live npm package metadata on 2026-08-07.

Blockers

  1. Generated current/latest version records are stale before merge.

    The PR records package-backed latest/current values from the 2026-08-05 tracker run, but live npm metadata has already advanced for multiple packages. Affected generated records include:

    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:134-167
    • artifacts/agent-version-tracker/summary.json:13-126
    • packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:7, :11, :74, :78, :114, :118, :144, :148, :286, :290, :459, :463, :533, :537

    Current npm latest values checked during review:

    Package PR records Current npm latest
    @ampcode/cli 0.0.1785906080-gd4ffa5 0.0.1786064749-gf2437d
    @anthropic-ai/claude-agent-sdk 0.3.222 0.3.223
    @anthropic-ai/claude-code 2.1.222 2.1.223
    @factory/cli 0.188.0 0.189.0
    @openai/codex 0.146.0 0.146.1
    @qwen-code/qwen-code 0.21.5 0.21.7
    opencode-ai 1.18.13 1.18.14

    This is a correctness blocker because the PR's purpose is to publish current upstream agent version data. Please rerun the tracker against current upstream metadata and regenerate the artifacts plus AgentVersion/EvidenceSource records before merge.

  2. QA is not passing.

    gh pr checks 1637 still reports Docs QA as FAILURE. The live-stack QA dispatched during this review also did not produce a passing terminal result:

    The review process has a strict gate: do not approve when QA failed or is inconclusive. Please get Docs QA green and re-run live-stack QA to a terminal passing result.

Major Finding

  1. Time-sensitive evidence wording is too absolute.

    packages/atlas/graph/catalog-meta/evidence-sources/upstream-current-2026-08-05.yaml:17-19 says npm metadata reports the Amp version as the latest dist-tag, and the Factory Droid evidence around :250 uses the same present-tense shape. These records are dated snapshots, so the wording should be an observed-at/captured-at fact rather than a global present-tense latest claim.

    Please rewrite these package metadata notes as timestamped observations, for example: "At observedAt, npm metadata reported ... as the latest dist-tag."

Risk Assessment

Risk level: risk:medium.

  • Risk: Atlas consumers ingest stale upstream version records as current/latest immediately after merge.

    • Pre-merge mitigation: rerun the tracker and compare package-backed records against live upstream metadata immediately before approval.
    • Deploy-time mitigation: no special rollout is needed for additive catalog data, but do not merge stale current records.
    • Post-merge mitigation: monitor the next scheduled tracker run for omitted versions.
  • Risk: red or inconclusive QA masks regressions in graph/catalog consumers.

    • Pre-merge mitigation: require Docs QA and live-stack QA to pass.
    • Deploy-time mitigation: rely on branch protection and do not merge while required checks are red.
    • Post-merge mitigation: keep docs freshness and live-stack runner capacity healthy so graph updates get reliable review signals.
  • Risk: historical evidence records become misleading as upstream dist-tags move.

    • Pre-merge mitigation: use observed-at/captured-at language for package metadata evidence.
    • Post-merge mitigation: consider a lightweight check or convention for generated evidence wording.

@a5c-ai

a5c-ai Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing yet. The focused adversarial live-stack workflow was dispatched, but it did not reach a terminal all-passing result within the 20-minute QA polling window.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31230762247
Branch/ref tested: agent-versions/daily-2026-08-05

Current job status at timeout

Job Status Result
Build All in_progress pending
Compute Matrix completed success

Focused matrix

Agent Model Mode Install Process mode
codex google-gemini31 interactive bp predefined
codex google-gemini31 interactive bp create
claude foundry-gpt55 interactive bp predefined
claude foundry-gpt55 bridged-hooks bp create
gemini google-gemini31 ni vanilla -
hermes foundry-gpt55 ni vanilla -

Selection rationale: PR #1637 changes generated Atlas AgentVersion graph records and catalog-meta EvidenceSource records consumed by BP/catalog paths. This matrix prioritizes BP predefined/create catalog consumers, includes a bridged-hooks BP path, and adds representative Gemini/Hermes vanilla adapter smoke coverage without running the full cross-product.

Verdict: not passed because the live-stack run had no terminal conclusion before the QA timeout. Re-check the Actions run after Build All completes.

@a5c-ai

a5c-ai Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing yet. The focused adversarial live-stack workflow was dispatched for the Atlas graph/catalog agent-version tracker update, but it did not reach a terminal all-passing result within the 20-minute QA polling window.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31230759159
Branch/ref tested: agent-versions/daily-2026-08-05

Current job status at timeout

Job Status Result
Compute Matrix completed success
Build All in_progress pending

Focused matrix

Agent Model Mode Install Process mode
codex google-gemini31 ni vanilla -
claude foundry-gpt55 ni vanilla -
gemini google-gemini31 ni vanilla -
codex google-gemini31 interactive bp predefined
claude foundry-gpt55 interactive bp create
hermes foundry-gpt55 bridged-hooks bp create

Scope rationale: PR #1637 changes Atlas graph AgentVersion records, catalog-meta EvidenceSource records, and generated tracker artifacts. This matrix focuses on BP/catalog-consuming paths across predefined/create and bridged-hooks modes, plus representative vanilla Codex/Claude/Gemini adapter sanity checks, rather than running the full transport/provider cross-product.

Verdict: not passed because Build All was still in progress at the QA timeout. Re-check the Actions run after it completes for the final live-stack result.

@a5c-ai

a5c-ai Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing yet. The focused adversarial live-stack workflow was dispatched, but it did not reach a terminal all-passing result within the 20-minute QA polling window.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31230788730
Branch/ref tested: agent-versions/daily-2026-08-05

Current job status at timeout

Job Status Result
Build All in_progress pending
Compute Matrix completed success

Focused matrix

Agent Model Mode Install Process mode
codex google-gemini31 ni vanilla -
claude anthropic-sonnet46 ni vanilla -
hermes foundry-gpt55 ni vanilla -
codex google-gemini31 interactive bp predefined
claude foundry-gpt55 bridged-hooks bp create

Rationale: PR #1637 changes Atlas AgentVersion graph records, catalog evidence records, and generated tracker artifacts. The matrix focuses on catalog-consuming BP paths with predefined and create process modes, includes bridged-hooks for hook-mediated BP behavior, and adds representative vanilla adapter checks across Google, Anthropic, Foundry, and Hermes/pip install coverage without running the full cross-product.

Verdict: not passed because the live-stack run had no terminal conclusion before the QA timeout. Re-check the Actions run after Build All completes.

@a5c-ai

a5c-ai Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing. The focused adversarial live-stack workflow was dispatched, but it remained queued through the 20-minute QA polling window, so no terminal live-stack pass/fail verdict is available.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31230799016
Branch/ref tested: agent-versions/daily-2026-08-05

Job status at timeout

Job Status Result
Compute Matrix completed success
Build All queued pending

Focused matrix

Agent Model Mode Install Process mode
codex google-gemini31 ni vanilla -
claude foundry-gpt55 ni vanilla -
codex google-gemini31 interactive bp predefined
claude foundry-gpt55 bridged-hooks bp create

Rationale: PR #1637 changes Atlas graph/catalog AgentVersion and EvidenceSource records plus generated tracker artifacts. This matrix targets BP catalog-consuming paths across predefined/create modes, includes hook-mediated BP behavior, and adds representative vanilla Codex/Google and Claude/Foundry adapter sanity checks without running the full cross-product.

Verdict: not passed because Build All did not start or complete before the QA timeout. Re-check the Actions run after runner capacity picks it up.

@a5c-ai

a5c-ai Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing yet. The focused adversarial live-stack workflow was dispatched, but it did not reach a terminal all-passing result within the 20-minute QA polling window.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31230795744
Branch/ref tested: agent-versions/daily-2026-08-05

Current job status at timeout

Job Status Result
Compute Matrix completed success
Build All in_progress pending

Focused matrix

Agent Model Mode Install Process mode
codex google-gemini31 interactive bp predefined
codex google-gemini31 interactive bp create
claude foundry-gpt55 bridged-hooks bp create
claude anthropic-sonnet46 ni vanilla -
gemini google-gemini31 ni vanilla -
hermes foundry-gpt55 ni vanilla -

Selection rationale: PR #1637 changes Atlas AgentVersion/catalog evidence records and tracker artifacts, so this adversarial matrix targets catalog/BP consumers plus representative raw harness adapters. It covers BP predefined/create paths, bridged hooks, direct Anthropic provider coverage, Gemini/Google coverage, and Hermes/foundry adapter sanity without running the full cross-product.

Verdict: not passed because the live-stack run had no terminal conclusion before the QA timeout. Re-check the Actions run after Build All completes.

@a5c-ai

a5c-ai Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing yet. The focused adversarial live-stack workflow was dispatched for PR #1637, but it remained queued through the 20-minute QA polling window, so no terminal all-passing result is available.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31230801588
Branch/ref tested: agent-versions/daily-2026-08-05

Job status at timeout

Job Status Result
Build All queued pending
Compute Matrix completed success

Focused matrix

Agent Model Mode Install Process mode
codex google-gemini31 ni vanilla -
claude foundry-gpt55 ni vanilla -
gemini google-gemini31 ni vanilla -
codex google-gemini31 interactive bp predefined
claude foundry-gpt55 interactive bp create
claude foundry-gpt55 bridged-hooks bp predefined

Selection rationale: PR #1637 changes Atlas graph/catalog AgentVersion and evidence-source records plus generated tracker artifacts. This focused matrix targets graph/catalog-consuming BP paths across predefined and create modes, includes a bridged-hooks plugin path, and adds representative vanilla adapter sanity checks for Codex, Claude, and Gemini without running the full cross-product.

Verdict: not passed because the live-stack run had no terminal conclusion before the QA timeout. Re-check the Actions run after runner capacity picks it up.

@a5c-ai

a5c-ai Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Adversarial review result: request changes.

I could not submit this as a formal request-changes review because GitHub rejects self-reviews from the PR authoring bot, but the decision is reject/not merge under the review process.

Blockers

  1. Generated current/latest version records are stale against live upstream package metadata.

    The PR records package-backed latest/current values from the 2026-08-05 tracker run, but live npm metadata has advanced again. Affected generated records include:

    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:134-167
    • packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:7, :11, :74, :78, :114, :118, :144, :148, :286, :290, :459, :463, :533, :537

    Current npm latest values checked during this review:

    Package PR records Current npm latest
    @ampcode/cli 0.0.1785906080-gd4ffa5 0.0.1786147648-g672f7d
    @anthropic-ai/claude-agent-sdk 0.3.222 0.3.224
    @anthropic-ai/claude-code 2.1.222 2.1.224
    @factory/cli 0.188.0 0.190.0
    @openai/codex 0.146.0 0.147.0
    @qwen-code/qwen-code 0.21.5 0.21.7
    opencode-ai 1.18.13 1.18.15
    @google/gemini-cli 0.53.1 0.54.4
    @oh-my-pi/pi-coding-agent 17.2.9 17.2.11
    @earendil-works/pi-coding-agent 0.83.0 0.84.1

    This is a correctness blocker because the PR's purpose is to publish current upstream agent version data. Merging known-stale current records would make Atlas/catalog data incorrect immediately.

    Fix: rerun the tracker against current upstream metadata immediately before merge, regenerate the artifacts plus AgentVersion/EvidenceSource records, and update/create tracking issues for newly observed versions.

  2. QA is not passing.

    gh pr checks 1637 still reports Docs QA as FAILURE; the other listed checks passed. The QA dispatch from this review completed as a dispatcher (qa-dispatch.yml run 31230607058), but downstream live-stack QA did not reach a terminal all-passing result. The posted QA reports show downstream runs including 31230759159 / 31230762247, where Compute Matrix passed but Build All was still in_progress at timeout.

    Under this review process, incomplete live-stack QA is not a passing verdict. Please get Docs QA green and rerun live-stack QA to a terminal passing result.

Major Finding

  1. Time-sensitive package evidence wording is too absolute.

    packages/atlas/graph/catalog-meta/evidence-sources/upstream-current-2026-08-05.yaml:18 says npm metadata reports the Amp version as the latest dist-tag, and :250 uses the same present-tense shape for Factory Droid. These are dated snapshots; the wording becomes false as soon as the dist-tag moves, which has already happened.

    Fix: phrase package metadata as observed-at evidence, for example: "At observedAt, npm metadata reported ... as the latest dist-tag."

Risk Assessment

Risk level: risk:medium.

  • Risk: Atlas/catalog consumers ingest stale upstream current agent-version records immediately after merge.

    • Pre-merge mitigation: rerun the tracker and compare every package-backed record against live npm metadata.
    • Deploy-time mitigation: do not merge stale current records.
    • Post-merge mitigation: monitor the next scheduled tracker run for omitted fast-follow releases.
  • Risk: red or inconclusive QA masks regressions in graph/catalog consumers.

    • Pre-merge mitigation: require Docs QA and live-stack QA to pass.
    • Deploy-time mitigation: keep branch protection/review policy blocking while required checks are red.
    • Post-merge mitigation: keep docs freshness and live-stack runner capacity healthy so graph updates get reliable review signals.
  • Risk: historical evidence records become misleading as upstream dist-tags move.

    • Pre-merge mitigation: use observed-at/captured-at language for package metadata evidence.
    • Post-merge mitigation: add a lightweight check or convention for generated evidence wording.

@a5c-ai

a5c-ai Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing. The adversarial live-stack workflow was dispatched successfully, but it remained queued through the 20-minute QA polling window, so no terminal all-passing verdict is available.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31230823829
Branch/ref tested: agent-versions/daily-2026-08-05

Job status at timeout

Job Status Result
Compute Matrix completed success
Build All queued pending

Focused adversarial matrix

Agent Model Mode Install Process mode
codex google-gemini31 interactive bp predefined
codex google-gemini31 interactive bp create
claude foundry-gpt55 interactive bp predefined
claude foundry-gpt55 bridged-hooks bp create
gemini google-gemini31 ni vanilla -
hermes foundry-gpt55 ni vanilla -
claude anthropic-sonnet46 ni vanilla -

Selection rationale: PR #1637 changes Atlas AgentVersion graph records, catalog evidence-source metadata, and generated tracker artifacts. The matrix focuses on BP catalog-consuming predefined/create paths, bridged-hooks behavior, and representative vanilla adapter/provider coverage without running the full cross-product.

Overall verdict: not passed because the live-stack run did not reach a terminal passing result before timeout. Re-check the Actions run after runner capacity picks it up.

@a5c-ai

a5c-ai Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing. The focused adversarial live-stack workflow was dispatched, but it did not reach a terminal all-passing result within the 20-minute QA polling window.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31230819817
Branch/ref tested: agent-versions/daily-2026-08-05

Current job status at timeout

Job Status Result
Compute Matrix completed success
Build All queued pending

Focused matrix

Agent Model Mode Install Process mode
codex google-gemini31 interactive bp predefined
claude foundry-gpt55 interactive bp create
codex foundry-gpt55 bridged-hooks bp create
hermes foundry-gpt55 ni vanilla -
gemini google-gemini31 ni vanilla -
claude anthropic-sonnet46 ni vanilla -

Selection rationale: PR #1637 changes Atlas AgentVersion graph records, catalog evidence-source records, and generated tracker artifacts. The matrix focuses on BP/plugin catalog consumers across predefined/create paths, includes a bridged-hooks BP path, and adds representative vanilla adapter checks for graph-tracked agents/providers without running the full cross-product.

Verdict: not passed because the live-stack run did not complete before the QA timeout. Re-check the Actions run after Build All starts and finishes.

@a5c-ai

a5c-ai Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Adversarial review result: request changes.

I reviewed the PR metadata, full diff, changed files from origin/pr/1637, current PR checks, the generated tracker artifacts and Atlas graph records, prior review/QA comments, and live npm package metadata on 2026-08-08.

Blockers

  1. Generated current/latest version records are stale against live upstream package metadata.

    The PR records package-backed latest/current values from the 2026-08-05 tracker run in artifacts/agent-version-tracker/upstream-targets-and-latest.json:132 and mirrors them through artifacts/agent-version-tracker/summary.json:2 plus the new current-version records, including:

    • packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:11
    • packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:78
    • packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:118
    • packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:148
    • packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:290
    • packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:320
    • packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:392
    • packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:463
    • packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:498
    • packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:537

    Current npm latest values checked during this review have advanced again:

    Package PR records Current npm latest
    @ampcode/cli 0.0.1785906080-gd4ffa5 0.0.1786147648-g672f7d
    @anthropic-ai/claude-agent-sdk 0.3.222 0.3.224
    @anthropic-ai/claude-code 2.1.222 2.1.224
    @factory/cli 0.188.0 0.190.0
    @google/gemini-cli 0.53.1 0.54.4
    @oh-my-pi/pi-coding-agent 17.2.9 17.2.11
    @earendil-works/pi-coding-agent 0.83.0 0.84.1
    @openai/codex 0.146.0 0.147.0
    @qwen-code/qwen-code 0.21.5 0.21.7
    opencode-ai 1.18.13 1.18.15

    Because this PR's purpose is to publish current upstream agent-version data, merging known-stale records would publish incorrect current catalog state. Please rerun/regenerate the tracker against current upstream metadata immediately before merge, refresh the artifacts plus AgentVersion/EvidenceSource records, and update/create tracking issues for newly observed versions.

  2. QA is not passing.

    gh pr checks 1637 still reports Docs QA as FAILURE. The failed job log shows docs:freshness detected stale generated docs:

    • docs/adapters/reference/04-agent-events.md: 56d > 45d
    • docs/harness-features-backlog/gaps/prompt-engineering/GAP-PROMPT-011.md: 56d > 45d

    I also dispatched the adversarial QA workflow for this review:

    • Run: 31230626431
    • Branch: agent-versions/daily-2026-08-05
    • Result after 25 polls through 2026-08-08T01:02:59Z: in_progress, job qa still in_progress, no conclusion.

    Under this review process, incomplete QA is not a passing verdict. Please get Docs QA green and rerun live-stack QA to a terminal passing result before approval.

Major Finding

  1. Time-sensitive package evidence wording is too absolute.

    packages/atlas/graph/catalog-meta/evidence-sources/upstream-current-2026-08-05.yaml:18, :167, and :250 use present-tense wording that npm metadata reports a version as the latest dist-tag. These are dated evidence records; that claim becomes misleading as soon as the dist-tag moves, which has already happened for several package-backed agents.

    Please phrase package metadata evidence as observed-at/captured-at facts, for example: "At observedAt, npm metadata reported ... as the latest dist-tag."

Verification

  • git diff --check origin/staging...HEAD: passed in an isolated PR worktree.
  • npm run verify:metadata: passed in the isolated PR worktree.
  • npm run build --workspace=@a5c-ai/atlas: inconclusive locally because the clean worktree did not have tsx installed; the PR's Lint, Tests, Package CI check is passing.

Risk Assessment

Risk level: risk:medium.

  • Risk: Atlas/catalog consumers ingest stale upstream version records as current/latest immediately after merge.

    • Pre-merge mitigation: rerun the tracker and compare package-backed records against live npm metadata immediately before approval.
    • Deploy-time mitigation: do not merge stale current records.
    • Post-merge mitigation: monitor the next scheduled tracker run for omitted fast-follow releases.
  • Risk: red or inconclusive QA masks regressions in graph/catalog consumers.

    • Pre-merge mitigation: require Docs QA and live-stack QA to pass.
    • Deploy-time mitigation: rely on branch protection and do not merge while required checks are red.
    • Post-merge mitigation: keep docs freshness and live-stack runner capacity healthy so graph updates get reliable review signals.
  • Risk: historical evidence records become misleading as upstream dist-tags move.

    • Pre-merge mitigation: use observed-at/captured-at language for package metadata evidence.
    • Post-merge mitigation: consider a lightweight check or convention for time-sensitive evidence wording.

Note: GitHub rejected a formal request-changes review from this authenticated actor because it owns the PR, so this is posted as a PR comment with the same request-changes decision.

@a5c-ai

a5c-ai Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Requesting changes. The PR still has active blockers under the adversarial review process.

Blockers

  1. Generated current/latest version records are stale against live upstream package metadata.

    The tracker artifacts were generated on 2026-08-05T07:38:33.458Z, but live npm metadata checked during this review on 2026-08-08 has moved for multiple package-backed records. Affected PR records include:

    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:134 records @ampcode/cli as 0.0.1785906080-gd4ffa5; live latest is 0.0.1786147648-g672f7d.
    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:138 records @anthropic-ai/claude-agent-sdk as 0.3.222; live latest is 0.3.224.
    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:142 records @anthropic-ai/claude-code as 2.1.222; live latest is 2.1.224.
    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:146 records @factory/cli as 0.188.0; live latest is 0.190.0.
    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:150 records @openai/codex as 0.146.0; live latest is 0.147.0.
    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:158 records @qwen-code/qwen-code as 0.21.5; live latest is 0.21.7.
    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:166 records opencode-ai as 1.18.13; live latest is 1.18.15.

    The stale values are mirrored into the new AgentVersion records, including packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:11, :78, :118, :148, :290, :320, :392, :463, :498, and :537. Because this PR publishes upstream current/latest catalog data, merging it would knowingly publish stale current records.

    Fix: rerun the tracker against current upstream metadata immediately before merge, regenerate the artifacts and AgentVersion/EvidenceSource records, then rerun metadata and Atlas verification.

  2. QA is not passing.

    gh pr checks 1637 still reports Docs QA as FAILURE. I also dispatched qa-dispatch.yml for branch agent-versions/daily-2026-08-05 and PR 1637:

    • Run: 31230624935
    • Result after 25 one-minute polls: still in_progress
    • Job qa remained in step Run a5c-ai/babysitter/packages/adapters/triggers@staging

    There is no terminal live-stack pass/fail result, so QA is inconclusive/not passed. Under this process, that is not approvable.

Major

packages/atlas/graph/catalog-meta/evidence-sources/upstream-current-2026-08-05.yaml:18 and :250 use present-tense latest wording: npm metadata "reports" those versions as latest. These are dated observations, and the dist-tags have already moved. Please rewrite package metadata evidence as observed-at facts, for example: "At observedAt, npm metadata reported ... as the latest dist-tag."

Risk Assessment

Risk level: risk:medium.

  • Risk: Atlas/catalog consumers ingest stale upstream current agent-version records immediately after merge.

    • Pre-merge mitigation: rerun the tracker and compare every package-backed record against live package metadata.
    • Deploy-time mitigation: do not merge stale current records.
    • Post-merge mitigation: monitor the next scheduled tracker run for omitted releases.
  • Risk: red or inconclusive QA masks regressions in graph/catalog consumers.

    • Pre-merge mitigation: require Docs QA and live-stack QA to reach terminal passing results.
    • Deploy-time mitigation: keep merge blocked while required checks are red.
    • Post-merge mitigation: keep docs freshness and live-stack runner capacity healthy.
  • Risk: historical evidence records become misleading as upstream dist-tags move.

    • Pre-merge mitigation: use observed-at/captured-at language for package metadata evidence.
    • Post-merge mitigation: add a lightweight convention or lint check for generated evidence wording.

@a5c-ai

a5c-ai Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Adversarial review result: request changes.

I reviewed the PR metadata, full diff, changed files from origin/pr/1637, existing comments/reviews, current PR checks, and live npm package metadata on 2026-08-08.

Blockers

  1. Generated current/latest version records are stale before merge.

    The PR records package-backed latest/current values from the 2026-08-05 tracker run in artifacts/agent-version-tracker/upstream-targets-and-latest.json:132, but live npm metadata has advanced again as of this review:

    Package PR records Current npm latest
    @ampcode/cli 0.0.1785906080-gd4ffa5 0.0.1786147648-g672f7d
    @anthropic-ai/claude-agent-sdk 0.3.222 0.3.224
    @anthropic-ai/claude-code 2.1.222 2.1.224
    @factory/cli 0.188.0 0.190.0
    @openai/codex 0.146.0 0.147.0
    @qwen-code/qwen-code 0.21.5 0.21.7
    opencode-ai 1.18.13 1.18.15

    These stale values are propagated into packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:11, :78, :118, :148, :290, :463, and :537 as currentVersion records. Because this PR's purpose is publishing current upstream version data, merging known-stale values would make Atlas/catalog data incorrect immediately.

    Fix: rerun the tracker against current upstream metadata, regenerate the tracker artifacts plus AgentVersion/EvidenceSource records, and rerun the Atlas build/metadata verification before merge.

  2. QA is not passing.

    gh pr checks 1637 still reports Docs QA as FAILURE. I also dispatched the requested live-stack QA run for this adversarial review:

    • Dispatcher run: 31230651907
    • Branch/ref tested: agent-versions/daily-2026-08-05
    • Result after 25 one-minute polls: still in_progress, job qa still in_progress, no terminal pass/fail result available

    Under this review process, failed or inconclusive QA is not a passing verdict. Please get Docs QA green and rerun live-stack QA to a terminal passing result.

Major Finding

  1. Time-sensitive evidence wording is too absolute.

    packages/atlas/graph/catalog-meta/evidence-sources/upstream-current-2026-08-05.yaml:18 says npm metadata reports @ampcode/cli as the latest dist-tag, and :250 says the same for @factory/cli. Those are dated observations, not timeless facts, and both are already false after the dist-tags moved.

    Fix: rewrite package metadata notes as observed-at facts, for example: "At observedAt, npm metadata reported ... as the latest dist-tag."

Risk Assessment

Risk level: risk:medium.

  • Risk: Atlas/catalog consumers ingest stale upstream version data as current/latest immediately after merge.

    • Pre-merge mitigation: rerun/regenerate tracker output and compare package-backed records against live npm metadata.
    • Deploy-time mitigation: do not merge stale current records.
    • Post-merge mitigation: monitor the next tracker run for omitted fast-follow releases.
  • Risk: quality signals are red or inconclusive.

    • Pre-merge mitigation: require Docs QA and live-stack QA to pass, or document a repository-approved exception.
    • Deploy-time mitigation: rely on branch protection and avoid merge while checks are red.
    • Post-merge mitigation: keep docs freshness and runner capacity healthy so graph updates get reliable review signals.
  • Risk: historical evidence records become misleading as moving dist-tags advance.

    • Pre-merge mitigation: use observed-at/captured-at language for package metadata evidence.
    • Post-merge mitigation: add a lightweight wording convention or lint for package metadata evidence.

@a5c-ai

a5c-ai Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing. The adversarial live-stack workflow was dispatched successfully, but it remained queued through the QA polling window, so no terminal all-passing verdict is available.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31286760457
Branch/ref tested: agent-versions/daily-2026-08-05

Job status at timeout

Job Status Result
Compute Matrix completed success
Build All queued pending

Focused adversarial matrix

Agent Model Mode Install Process mode
codex google-gemini31 interactive bp predefined
codex google-gemini31 interactive bp create
claude foundry-gpt55 interactive bp predefined
claude foundry-gpt55 bridged-hooks bp create
gemini google-gemini31 ni vanilla -
hermes foundry-gpt55 ni vanilla -
claude anthropic-sonnet46 ni vanilla -

Selection rationale: PR #1637 changes Atlas AgentVersion graph records, catalog evidence-source metadata, and generated tracker artifacts. The matrix focuses on BP graph/catalog-consuming predefined/create paths, includes a bridged-hooks BP path, and adds representative vanilla adapter/provider checks without running the full cross-product.

Overall verdict: not passed because the live-stack run did not reach a terminal passing result before timeout. This does not clear the existing stale current/latest blockers or the Docs QA failure noted in review.

@a5c-ai

a5c-ai Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing. The adversarial live-stack workflow was dispatched successfully, but it did not reach a terminal all-passing result within the 20-minute QA polling window.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31286751378
Branch/ref tested: agent-versions/daily-2026-08-05

Final polling note: the run was still in_progress at poll 20 (2026-08-09T01:01:57Z); the final poll hit the GitHub installation API rate limit, so the verdict is based on the last successful observed status inside the timeout window.

Job status at timeout

Job Status Result
Compute Matrix completed success
Build All in_progress pending

Focused adversarial matrix

Agent Model Mode Install Process mode
codex google-gemini31 interactive bp predefined
codex google-gemini31 interactive bp create
claude foundry-gpt55 interactive bp predefined
claude foundry-gpt55 bridged-hooks bp create
gemini google-gemini31 ni vanilla -
hermes foundry-gpt55 ni vanilla -
claude anthropic-sonnet46 ni vanilla -

Selection rationale: PR #1637 changes Atlas AgentVersion graph records, catalog evidence-source metadata, and generated tracker artifacts. The matrix focuses on BP/plugin catalog consumers across predefined/create paths, includes bridged-hooks coverage, and adds representative vanilla adapter/provider checks without running the full cross-product.

Overall verdict: not passed because the live-stack run did not complete before the QA timeout.

@a5c-ai

a5c-ai Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing. The adversarial live-stack workflow was dispatched, but it did not reach a terminal all-passing result within the QA polling window.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31286782000
Branch/ref tested: agent-versions/daily-2026-08-05

Polling result

Check Status Result
workflow_dispatch run queued through polling window queued timed out
final status check blocked GitHub API rate limited

The run was still queued on successful polls from 2026-08-09T00:43:50Z through 2026-08-09T01:01:58Z. The final poll at 2026-08-09T01:02:59Z was blocked by a GitHub API rate-limit response, so no terminal job conclusions were available.

Focused adversarial matrix

Agent Model Mode Install Process mode
codex google-gemini31 interactive bp predefined
codex google-gemini31 interactive bp create
claude foundry-gpt55 interactive bp predefined
claude foundry-gpt55 bridged-hooks bp create
gemini google-gemini31 ni vanilla -
hermes foundry-gpt55 ni vanilla -
claude anthropic-sonnet46 ni vanilla -

Selection rationale: PR #1637 changes Atlas AgentVersion graph records, catalog evidence-source metadata, and generated tracker artifacts. This matrix focuses on BP catalog-consuming paths across predefined/create process modes, includes a bridged-hooks BP path, and adds representative vanilla adapter/provider checks for graph-tracked agents without running the full cross-product.

Overall verdict: not passed because live-stack did not reach a terminal passing result within the process window.

@a5c-ai

a5c-ai Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing. The adversarial live-stack workflow was dispatched successfully, but it stayed queued for 19 one-minute polling attempts. The 20th poll was blocked by the GitHub API rate limit, so no terminal all-passing verdict is available.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31286781250
Branch/ref tested: agent-versions/daily-2026-08-05

Job status at timeout

Job Status Result
workflow_dispatch run queued through 19 polling attempts; poll 20 blocked by GitHub API rate limit queued timed_out_queued

Focused adversarial matrix

Agent Model Mode Install Process mode
codex google-gemini31 interactive bp predefined
codex google-gemini31 interactive bp create
claude foundry-gpt55 interactive bp predefined
claude foundry-gpt55 bridged-hooks bp create
gemini google-gemini31 ni vanilla -
hermes foundry-gpt55 ni vanilla -
claude anthropic-sonnet46 ni vanilla -

Selection rationale: PR #1637 changes Atlas AgentVersion graph records, catalog evidence-source metadata, and generated tracker artifacts. The matrix targets BP/plugin paths that consume the catalog in both predefined and create process modes, includes bridged-hooks coverage for hook transport, and adds representative vanilla adapter/provider checks for graph-tracked agents without running the full cross-product.

Overall verdict: not passed because the live-stack run did not reach a terminal passing result before the QA polling window ended.

@a5c-ai

a5c-ai Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing. The adversarial live-stack workflow was dispatched successfully, but it remained queued through the 20-minute QA polling window, so no terminal all-passing verdict is available.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31286760702
Branch/ref tested: agent-versions/daily-2026-08-05

Job status at timeout

Job Status Result
workflow_dispatch run queued past 20 minute QA timeout queued timed_out_queued

Note: the final post-timeout GitHub summary request hit an installation API rate limit, but the 20 scheduled status polls all returned queued with no conclusion.

Focused adversarial matrix

Agent Model Mode Install Process mode
codex google-gemini31 interactive bp predefined
codex google-gemini31 interactive bp create
claude foundry-gpt55 interactive bp predefined
claude foundry-gpt55 bridged-hooks bp create
gemini google-gemini31 ni vanilla -
hermes foundry-gpt55 ni vanilla -
claude anthropic-sonnet46 ni vanilla -

Selection rationale: PR #1637 changes Atlas AgentVersion graph records, catalog evidence-source metadata, and generated tracker artifacts. The matrix focuses on BP catalog-consuming predefined/create paths, bridged-hooks behavior, and representative vanilla adapter/provider coverage without running the full cross-product.

Overall verdict: not passed because the live-stack run did not reach a terminal passing result before timeout. Re-check the Actions run after runner capacity picks it up.

@a5c-ai

a5c-ai Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing. The focused adversarial live-stack workflow was dispatched successfully, but it did not reach a terminal all-passing result within the 20-poll QA window.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31286758023
Branch/ref tested: agent-versions/daily-2026-08-05

Current job status at timeout

Job Status Result
Compute Matrix completed success
Build All in_progress pending

Last successful poll: 2026-08-09T01:02:18Z. A final refresh after timeout hit the GitHub API rate limit for the installation, so the table above reflects the last successful poll during the QA window.

Focused adversarial matrix

Agent Model Mode Install Process mode
codex google-gemini31 interactive bp predefined
codex google-gemini31 interactive bp create
claude foundry-gpt55 interactive bp predefined
claude foundry-gpt55 interactive bp create
codex foundry-gpt55 bridged-hooks bp create
gemini google-gemini31 ni vanilla -
hermes foundry-gpt55 ni vanilla -
claude anthropic-sonnet46 ni vanilla -

Selection rationale: PR #1637 changes Atlas AgentVersion graph records, catalog evidence-source metadata, and generated tracker artifacts. The matrix focuses on BP/plugin catalog-consuming predefined/create paths, includes a bridged-hooks BP path, and adds representative vanilla adapter/provider coverage without running the full cross-product.

Overall verdict: not passed because the live-stack run did not complete before the QA timeout.

@a5c-ai

a5c-ai Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Live-stack QA

Result: incomplete / not passing. The adversarial live-stack workflow was dispatched successfully, but it did not reach a terminal all-passing result within the 20-minute QA polling window.

Run: https://github.com/a5c-ai/babysitter/actions/runs/31286783037
Branch/ref tested: agent-versions/daily-2026-08-05

Last observed job status

Job Status Result
Compute Matrix completed success
Build All queued pending
Polling final status blocked api_rate_limited_after_20_minute_window

Focused adversarial matrix

Agent Model Mode Install Process mode
codex google-gemini31 interactive bp predefined
codex google-gemini31 interactive bp create
claude foundry-gpt55 interactive bp predefined
claude foundry-gpt55 bridged-hooks bp create
gemini google-gemini31 ni vanilla -
hermes foundry-gpt55 ni vanilla -
claude anthropic-sonnet46 ni vanilla -

Selection rationale: PR #1637 changes Atlas AgentVersion graph records, catalog evidence-source metadata, and generated tracker artifacts. The matrix focuses on BP catalog-consuming predefined/create paths, bridged-hooks behavior, and representative vanilla adapter/provider coverage without running the full cross-product.

Overall verdict: not passed. The live-stack run was still queued at the last successful poll, and the final status polls hit the GitHub installation API rate limit, so no terminal live-stack pass verdict is available from this QA run.

@a5c-ai

a5c-ai Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Adversarial review result: request changes.

I reviewed PR metadata, the full diff, changed files from origin/pr-1637, current PR checks, prior review/QA comments, and live npm package metadata on 2026-08-09.

Blockers

  1. Generated current/latest version records are stale against live upstream package metadata.

    The PR records package-backed latest/current values from the 2026-08-05 tracker run in artifacts/agent-version-tracker/upstream-targets-and-latest.json, including:

    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:134 records @ampcode/cli as 0.0.1785906080-gd4ffa5; live npm latest is 0.0.1786233956-g40887a.
    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:138 records @anthropic-ai/claude-agent-sdk as 0.3.222; live npm latest is 0.3.226.
    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:142 records @anthropic-ai/claude-code as 2.1.222; live npm latest is 2.1.226.
    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:146 records @factory/cli as 0.188.0; live npm latest is 0.190.0.
    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:150 records @openai/codex as 0.146.0; live npm latest is 0.147.0.
    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:158 records @qwen-code/qwen-code as 0.21.5; live npm latest is 0.21.8.
    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:166 records opencode-ai as 1.18.13; live npm latest is 1.18.15.

    Those stale values are propagated into the new AgentVersion records, including:

    • packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:11
    • packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:78
    • packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:118
    • packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:148
    • packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:290
    • packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:463
    • packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:537

    This PR's purpose is to publish upstream current/latest agent-version data. Merging known-stale currentVersion records would make Atlas/catalog consumers ingest incorrect current state immediately.

    Fix: rerun the tracker against current upstream metadata immediately before merge, regenerate the tracker artifacts plus AgentVersion/EvidenceSource records, and rerun Atlas build/metadata verification.

  2. QA is not passing.

    gh pr checks 1637 reports Docs QA as FAILURE. I also dispatched fresh QA for this review:

    • Run: 31286626395
    • Branch/ref tested: agent-versions/daily-2026-08-05
    • Last successful poll: 2026-08-09T01:02:44Z
    • Status at that poll: workflow in_progress, job qa in_progress, step Run a5c-ai/babysitter/packages/adapters/triggers@staging still in_progress
    • Subsequent polling was blocked by GitHub HTTP 403 API rate limit for the installation, so no terminal pass/fail result could be read.

    Under this review process, incomplete QA is not a passing verdict. Please get Docs QA green and rerun live-stack QA to a terminal passing result before approval.

Major Finding

  1. Time-sensitive package evidence wording is too absolute.

    packages/atlas/graph/catalog-meta/evidence-sources/upstream-current-2026-08-05.yaml:18 says npm metadata reports @ampcode/cli as the latest dist-tag, and packages/atlas/graph/catalog-meta/evidence-sources/upstream-current-2026-08-05.yaml:250 says the same for @factory/cli. These are dated observations, not timeless facts, and both claims are already false after dist-tags moved.

    Fix: rewrite package metadata notes as observed-at/captured-at facts, for example: "At observedAt, npm metadata reported ... as the latest dist-tag."

Verification

  • git diff --check origin/staging..origin/pr-1637: passed.
  • npm run verify:metadata: not conclusive for this PR in the current checkout because it fails on the unrelated .agents/plugins/marketplace.json metadata issue noted in the PR body.

Risk Assessment

Risk level: risk:medium.

  • Risk: Atlas/catalog consumers ingest stale upstream version data as current/latest immediately after merge.

    • Pre-merge mitigation: rerun/regenerate tracker output and compare all package-backed records against live package metadata.
    • Deploy-time mitigation: do not merge stale current records.
    • Post-merge mitigation: monitor the next scheduled tracker run for omitted fast-follow releases.
  • Risk: red or inconclusive QA masks regressions in graph/catalog consumers.

    • Pre-merge mitigation: require Docs QA and live-stack QA to pass, or document a repository-approved exception.
    • Deploy-time mitigation: rely on branch protection and avoid merge while checks are red.
    • Post-merge mitigation: keep docs freshness and live-stack runner capacity healthy.
  • Risk: historical package metadata evidence becomes misleading as moving dist-tags advance.

    • Pre-merge mitigation: use observed-at/captured-at wording for package metadata evidence.
    • Post-merge mitigation: consider a lightweight convention or lint check for time-sensitive evidence wording.

@a5c-ai

a5c-ai Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Adversarial review result: request changes.

I reviewed PR metadata, the full diff, changed files from origin/pr/1637, current PR checks, Docs QA logs, live npm metadata on 2026-08-09 UTC, prior review/QA comments, and the QA dispatch created for this review.

Blockers

  1. Generated current/latest version records are stale against live upstream package metadata.

    The PR records package-backed latest/current values from the 2026-08-05 tracker run in artifacts/agent-version-tracker/upstream-targets-and-latest.json:132, including:

    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:134 records @ampcode/cli as 0.0.1785906080-gd4ffa5; live latest is 0.0.1786233956-g40887a.
    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:138 records @anthropic-ai/claude-agent-sdk as 0.3.222; live latest is 0.3.226.
    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:142 records @anthropic-ai/claude-code as 2.1.222; live latest is 2.1.226.
    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:146 records @factory/cli as 0.188.0; live latest is 0.190.0.
    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:150 records @openai/codex as 0.146.0; live latest is 0.147.0.
    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:158 records @qwen-code/qwen-code as 0.21.5; live latest is 0.21.8.
    • artifacts/agent-version-tracker/upstream-targets-and-latest.json:166 records opencode-ai as 1.18.13; live latest is 1.18.15.

    Additional package-backed records in the new AgentVersion YAML are also stale: @google/gemini-cli is now 0.54.4 vs PR 0.53.1, @oh-my-pi/pi-coding-agent is now 17.2.11 vs PR 17.2.9, and @earendil-works/pi-coding-agent is now 0.84.1 vs PR 0.83.0.

    These values are propagated into current records in packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml, including lines 11, 78, 118, 148, 290, 320, 392, 463, 498, and 537. Because this PR publishes upstream current/latest catalog data, merging known-stale current records would make Atlas/catalog state incorrect immediately.

    Fix: rerun the tracker against current upstream metadata immediately before merge, regenerate the tracker artifacts plus AgentVersion/EvidenceSource records, and update/create tracking issues for newly observed versions.

  2. QA is not passing.

    gh pr checks 1637 still reports Docs QA as FAILURE, and GitHub reports the PR merge state as UNSTABLE. The failed Docs QA job log shows docs:freshness detected stale generated documentation:

    • docs/adapters/reference/04-agent-events.md: generated doc is stale (56d > 45d)
    • docs/harness-features-backlog/gaps/prompt-engineering/GAP-PROMPT-011.md: generated doc is stale (56d > 45d)

    I also dispatched qa-dispatch.yml for this review:

    • Run: 31286641829
    • Branch/ref tested: agent-versions/daily-2026-08-05
    • Result: no terminal passing verdict observed. Last successful polls at 2026-08-09T01:01:00Z and 2026-08-09T01:02:00Z still showed the run in_progress at Run a5c-ai/babysitter/packages/adapters/triggers@staging; later polls hit GitHub API rate limits.

    Under this process, incomplete QA is not passing. Please get Docs QA green and rerun live-stack QA to a terminal passing result before approval.

Major Finding

  1. Time-sensitive package evidence wording is too absolute.

    packages/atlas/graph/catalog-meta/evidence-sources/upstream-current-2026-08-05.yaml:18, :167, :250, and :279 use present-tense wording that npm metadata reports a version as the latest dist-tag. These records are dated observations, and several dist-tags have already moved, so the wording becomes false after publication.

    Fix: phrase package metadata evidence as observed-at facts, for example: "At observedAt, npm metadata reported ... as the latest dist-tag."

Verification

  • git diff --check origin/staging...origin/pr/1637: passed.
  • Current PR checks: Docs QA failed; Lint, Tests, Package, Workspace Coverage, and Observer Dashboard passed.
  • Docs QA log confirms docs:freshness failed on the two stale generated docs listed above.
  • Live npm latest checks were run during this review on 2026-08-09 UTC.

Risk Assessment

Risk level: risk:medium.

  • Risk: Atlas/catalog consumers ingest stale upstream version data as current/latest immediately after merge.

    • Pre-merge mitigation: rerun/regenerate tracker output and compare package-backed records against live npm metadata.
    • Deploy-time mitigation: do not merge stale current records.
    • Post-merge mitigation: monitor the next tracker run for omitted fast-follow releases.
  • Risk: red or incomplete QA masks regressions in graph/catalog consumers.

    • Pre-merge mitigation: require Docs QA and live-stack QA to reach terminal passing results.
    • Deploy-time mitigation: rely on branch protection and avoid merge while checks are red.
    • Post-merge mitigation: keep docs freshness and runner capacity healthy.
  • Risk: historical evidence records become misleading as moving dist-tags advance.

    • Pre-merge mitigation: use observed-at/captured-at language for package metadata evidence.
    • Post-merge mitigation: add a lightweight convention or lint for generated package metadata wording.

Note: GitHub rejected a formal request-changes review from this authenticated actor because it owns the PR, so this comment carries the same request-changes decision.

@a5c-ai

a5c-ai Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Adversarial review result: request changes.

I reviewed the PR metadata, full diff, changed files from origin/pr/1637 in /tmp/pr-1637-review-01KZHZ-2, current PR checks, existing review/QA comments, local metadata verification, and live npm package metadata on 2026-08-09.

Blockers

  1. Generated current/latest version records are stale against live upstream package metadata.

    The PR records package-backed latest/current values from the 2026-08-05 tracker run in artifacts/agent-version-tracker/upstream-targets-and-latest.json:132, with generated timestamp artifacts/agent-version-tracker/summary.json:2. Live npm latest values checked during this review have moved again:

    Package PR records Current npm latest
    @ampcode/cli 0.0.1785906080-gd4ffa5 0.0.1786233956-g40887a
    @anthropic-ai/claude-agent-sdk 0.3.222 0.3.226
    @anthropic-ai/claude-code 2.1.222 2.1.226
    @factory/cli 0.188.0 0.190.0
    @google/gemini-cli 0.53.1 0.54.4
    @oh-my-pi/pi-coding-agent 17.2.9 17.2.11
    @earendil-works/pi-coding-agent 0.83.0 0.84.1
    @openai/codex 0.146.0 0.147.0
    @qwen-code/qwen-code 0.21.5 0.21.8
    opencode-ai 1.18.13 1.18.15

    These stale values are mirrored into new AgentVersion records, including packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml:11, :78, :118, :148, :290, :320, :463, :498, and :537. Because this PR's purpose is publishing current/latest upstream agent-version catalog data, merging known-stale records would make Atlas/catalog state incorrect immediately.

    Fix: rerun the tracker against current upstream metadata immediately before merge, regenerate the tracker artifacts plus AgentVersion/EvidenceSource records, and rerun metadata/Atlas verification.

  2. QA is not passing.

    gh pr checks 1637 still reports Docs QA as FAILURE. The failed job log shows docs:freshness detected stale generated docs:

    • docs/adapters/reference/04-agent-events.md: generated doc is stale (56d > 45d)
    • docs/harness-features-backlog/gaps/prompt-engineering/GAP-PROMPT-011.md: generated doc is stale (56d > 45d)

    I also dispatched QA for this review:

    • Run: 31286629673
    • Branch/ref tested: agent-versions/daily-2026-08-05
    • Poll result: polls 1-24 showed the dispatcher queued then in progress in Run a5c-ai/babysitter/packages/adapters/triggers@staging; poll 25 hit the GitHub API rate limit, so no terminal all-passing verdict could be confirmed.

    Under this review process, failed or inconclusive QA is not passing. Please get Docs QA green and rerun live-stack QA to a terminal passing result, or document a repository-approved exception before merge.

Major

packages/atlas/graph/catalog-meta/evidence-sources/upstream-current-2026-08-05.yaml:18 and :250 use present-tense latest wording for package metadata. These are dated observations, and the dist-tags have already moved. Please rewrite package metadata notes as observed-at/captured-at facts, for example: "At observedAt, npm metadata reported ... as the latest dist-tag."

Verification

  • git diff --check origin/staging...HEAD: passed in the isolated PR worktree.
  • npm run verify:metadata: passed in the isolated PR worktree.
  • npm run build --workspace=@a5c-ai/atlas: inconclusive locally because the clean worktree did not have tsx installed; the PR's Lint, Tests, Package check is passing.

Risk Assessment

Risk level: risk:medium.

  • Risk: Atlas/catalog consumers ingest stale upstream version data as current/latest immediately after merge.

    • Pre-merge mitigation: rerun/regenerate tracker output and compare package-backed records against live npm metadata.
    • Deploy-time mitigation: do not merge stale current records.
    • Post-merge mitigation: monitor the next tracker run for omitted fast-follow releases.
  • Risk: red or inconclusive QA masks regressions in graph/catalog consumers.

    • Pre-merge mitigation: require Docs QA and live-stack QA to pass, or document a repository-approved exception.
    • Deploy-time mitigation: rely on branch protection and avoid merge while checks are red.
    • Post-merge mitigation: keep docs freshness and runner capacity healthy so graph updates get reliable review signals.
  • Risk: historical evidence records become misleading as upstream dist-tags move.

    • Pre-merge mitigation: use observed-at/captured-at language for package metadata evidence.
    • Post-merge mitigation: add a lightweight wording convention or lint check for generated package-metadata evidence.

Note: GitHub rejected a formal request-changes review from this authenticated actor because it owns the PR, so this is posted as a PR comment with the same request-changes decision.

@a5c-ai

a5c-ai Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Adversarial review result: request changes.

Blockers remain:

  1. Generated current/latest records are stale against live npm metadata checked on 2026-08-09. Examples: @ampcode/cli PR 0.0.1785906080-gd4ffa5 vs latest 0.0.1786233956-g40887a; @anthropic-ai/claude-code PR 2.1.222 vs latest 2.1.226; @qwen-code/qwen-code PR 0.21.5 vs latest 0.21.8; opencode-ai PR 1.18.13 vs latest 1.18.15. The stale values are in artifacts/agent-version-tracker/upstream-targets-and-latest.json:134, :138, :142, :146, :150, :158, :166 and propagated into packages/atlas/graph/agent-stack/agent-versions/upstream-current-2026-08-05.yaml currentVersion records.

  2. QA is not passing. Docs QA is still failed. Dispatched QA run 31286631945 did not reach a terminal passing result in the polling window; it remained in progress through poll 24, then poll 25 hit the GitHub App API rate limit.

Major: packages/atlas/graph/catalog-meta/evidence-sources/upstream-current-2026-08-05.yaml:18 and :250 use present-tense "latest dist-tag" wording for dated observations. Please rewrite those as observed-at/captured-at facts.

Risk level: risk:medium. Do not merge stale current catalog records; rerun the tracker against live upstream metadata, get Docs QA and live-stack QA to terminal passing, and regenerate the evidence wording before approval.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants