Skip to content

fix(anvil): harden workspace checks - #107

Merged
martin-kolinek merged 23 commits into
mainfrom
u/makolnek/anvil-generic-check-fixes
Sep 1, 2026
Merged

fix(anvil): harden workspace checks#107
martin-kolinek merged 23 commits into
mainfrom
u/makolnek/anvil-generic-check-fixes

Conversation

@martin-kolinek

@martin-kolinek martin-kolinek commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

🤖 This hardens generic cargo-anvil checks surfaced while adopting Anvil in ox-sdk and by runtime-analysis failures in ox-tools.

Summary

  • use pinned cargo-each to run rustfmt once per workspace member, avoiding platform command-line limits and hand-written metadata loops while reporting all failing members
  • make --dry-run detect exact .anvil.lock drift—including stale checksums, missing inventory, catalog metadata, and formatting—and run the dogfood regeneration gate on every pull request
  • keep Loom exploration exhaustive locally and in CI while reducing redundant symmetric participants in cargo-gamma's models so their state spaces remain tractable
  • invoke cargo-bolero with its supported release-profile syntax and enable the compile-time feature required by its libfuzzer adapter
  • run external-types for every library crate, including publish = false crates with in-workspace consumers, while semver skips publish = false crates because they have no registry baseline
  • add recipe contract coverage and refresh generated snapshots

Cargo Sort continues to enforce both dependency ordering and its manifest formatting convention with --check-format.

Validation

  • cargo test -p cargo-anvil
  • cargo clippy -p cargo-anvil --all-targets --all-features -- -D warnings
  • dry-run regression coverage for manifest-only and formatting-only lockfile drift
  • exhaustive cargo-gamma-lib Loom models with no global preemption bound
  • cargo-anvil documentation, README, and spelling checks
  • cargo-each dry-run over the ox-tools workspace
  • per-package formatting in ox-tools and ox-sdk through the Substrate backport

@codecov-commenter

Codecov Comments Bot (codecov-commenter) commented Aug 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.7%. Comparing base (6054135) to head (77d02b1).
⚠️ Report is 1 commits behind head on main.

❌ Your project status has failed because the head coverage (97.7%) is below the target coverage (100.0%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
@@           Coverage Diff           @@
##            main    #107     +/-   ##
=======================================
- Coverage   97.7%   97.7%   -0.1%     
=======================================
  Files        286     286             
  Lines      62460   62438     -22     
=======================================
- Hits       61032   61010     -22     
  Misses      1428    1428             
Flag Coverage Δ
linux 97.6% <100.0%> (-0.1%) ⬇️
linux-arm 97.6% <100.0%> (+<0.1%) ⬆️
scheduled ?
windows 97.8% <100.0%> (-0.1%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

martin-kolinek and others added 5 commits August 26, 2026 16:03
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@martin-kolinek
martin-kolinek force-pushed the u/makolnek/anvil-generic-check-fixes branch from e42622a to 3fdad6f Compare August 26, 2026 14:05
Comment thread crates/cargo-anvil/templates/justfiles/anvil/checks/fmt.just Outdated
Comment thread crates/cargo-anvil/templates/justfiles/anvil/checks/external-types.just Outdated
Comment thread crates/cargo-anvil/templates/justfiles/anvil/checks/cargo-sort.just Outdated
Comment thread crates/cargo-anvil/docs/design/local.md Outdated
Comment thread crates/cargo-anvil/tests/recipe_contracts.rs Outdated
Comment thread crates/cargo-anvil/tests/recipe_contracts.rs Outdated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4112f005-775f-4c4b-88ed-db8f0c5d1cf1
@martin-kolinek
martin-kolinek marked this pull request as ready for review August 28, 2026 08:18
Copilot AI lite review requested due to automatic review settings August 28, 2026 08:18
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4112f005-775f-4c4b-88ed-db8f0c5d1cf1

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request hardens cargo-anvil’s workspace-oriented checks by tightening the “what should be checked” contract (publishable libraries only where appropriate) and by delegating workspace iteration for formatting to a pinned cargo-each, improving robustness on large workspaces and across platforms.

Changes:

  • Pin and install/validate cargo-each, and use it to run rustfmt once per workspace member manifest with --keep-going.
  • Update external-types and semver-check to skip publish = false libraries (no registry consumer contract), and improve skip/exclusion messaging.
  • Update design docs, recipe-contract tests, snapshots, and the .anvil.lock catalog checksum to reflect the hardened behavior.

Reviewed changes

Copilot reviewed 21 out of 22 changed files in this pull request and generated no comments.

Show a summary per file
File Description
justfiles/anvil/versions.just Adds pinned cargo_each_version.
justfiles/anvil/tools.just Adds install/validate recipes for pinned cargo-each.
justfiles/anvil/helpers.just Clarifies modified-tier semantics as a skip gate (no package splicing).
justfiles/anvil/checks/semver-check.just Skips non-publishable library crates and updates messaging.
justfiles/anvil/checks/fmt.just Switches to cargo each --workspace --keep-going per-manifest cargo fmt.
justfiles/anvil/checks/external-types.just Skips publish = false libraries; reports exclusions; checks publishable libs only.
justfiles/anvil/checks/cargo-sort.just Tightens/clarifies rationale for --check-format and --grouped.
crates/cargo-anvil/tests/snapshots/snapshots__local_only.snap Refreshes generated snapshot output for updated recipes/docs.
crates/cargo-anvil/tests/snapshots/snapshots__github_backend.snap Refreshes generated snapshot output for updated recipes/docs.
crates/cargo-anvil/tests/snapshots/snapshots__ado_backend.snap Refreshes generated snapshot output for updated recipes/docs.
crates/cargo-anvil/tests/recipe_contracts.rs Adds recipe contract tests for fmt via cargo-each and publish filtering behavior.
crates/cargo-anvil/templates/justfiles/anvil/versions.just Mirrors cargo_each_version pin in templates.
crates/cargo-anvil/templates/justfiles/anvil/tools.just Mirrors cargo-each tool recipes in templates.
crates/cargo-anvil/templates/justfiles/anvil/helpers.just Mirrors helper-doc updates in templates.
crates/cargo-anvil/templates/justfiles/anvil/checks/semver-check.just Mirrors semver publish filtering in templates.
crates/cargo-anvil/templates/justfiles/anvil/checks/fmt.just Mirrors fmt delegation to cargo-each in templates.
crates/cargo-anvil/templates/justfiles/anvil/checks/external-types.just Mirrors external-types publish filtering/reporting in templates.
crates/cargo-anvil/templates/justfiles/anvil/checks/cargo-sort.just Mirrors cargo-sort comment updates in templates.
crates/cargo-anvil/src/anvil/artifacts/justfile.rs Ensures versions.just contract includes cargo_each_version.
crates/cargo-anvil/docs/design/local.md Updates local design docs to match modified-tier and fmt execution model.
crates/cargo-anvil/docs/design/checks.md Updates check catalog docs for fmt, cargo-sort, semver-check, and external-types.
.anvil.lock Updates catalog checksum and file checksums for the changed catalog files.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot AI review requested due to automatic review settings August 28, 2026 08:22

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 22 changed files in this pull request and generated 2 comments.

Suppressed comments (7)

Previously missed (5) — in code that hasn't changed since the last review.

crates/cargo-anvil/docs/design/local.md:469

  • The anvil-fmt example uses cargo fmt without the pinned nightly toolchain, but the actual recipe invokes rustfmt via the pinned nightly (cargo '+{{ rust_nightly }}' fmt ...). This example should include the +<pinned-nightly> prefix to avoid suggesting a different/incorrect invocation.
    cargo each --workspace --keep-going -- \
        cargo fmt --manifest-path '{manifest}' --check

justfiles/anvil/checks/external-types.just:32

  • This comment says "private libraries", but the code is specifically partitioning on Cargo metadata publish being an empty array (i.e. publish = false / non-publishable). Calling these "private" is ambiguous (it could be read as "published to a private registry").
    # Build pkg-name -> manifest-path maps for publishable and private libraries.

crates/cargo-anvil/templates/justfiles/anvil/checks/external-types.just:32

  • This comment says "private libraries", but the code is specifically partitioning on Cargo metadata publish being an empty array (i.e. publish = false / non-publishable). Calling these "private" is ambiguous (it could be read as "published to a private registry").
    # Build pkg-name -> manifest-path maps for publishable and private libraries.

crates/cargo-anvil/docs/design/checks.md:173

  • The semver-check row says "Private and bin-only crates..." but the implemented filter is specifically publish = false (Cargo metadata publish: []). Consider spelling that out to match the actual semantics and avoid ambiguity with "private registry" crates.
| `semver-check`                 | `cargo semver-checks --baseline-rev <baseline>` per affected publishable library crate. Private and bin-only crates have no registry consumer contract and are skipped. The PR target is the baseline. Exit 100 is a completed check with deny-level findings; exit 101 or another nonzero status means the comparison was inconclusive. Both outcomes write `target/anvil/comments/semver.md` and remain advisory, matching the repository's native `semver` job (`continue-on-error: true`). Proven rename and bin→lib transitions with no comparable baseline are skipped without a comment. Anvil preflight failures such as invalid current-workspace metadata or an unavailable baseline ref still fail because the recipe cannot establish what to compare. | oxidizer-github |

crates/cargo-anvil/tests/recipe_contracts.rs:687

  • This test name uses "private", but the scenario being exercised is publish = false / non-publishable crates. Renaming to "non_publishable" would better match the behavior and avoid confusion with crates published to a private registry.

This issue also appears in the following locations of the same file:

  • line 715
  • line 770
fn external_types_checks_every_publishable_library_and_reports_private_ones() {

crates/cargo-anvil/tests/recipe_contracts.rs:715

  • This assertion message says "private" but the filtered-out case is publish = false / non-publishable libraries. Using the same terminology as the recipe output ("non-publishable") makes failures easier to understand.
        "private library filtering failed\nstdout:\n{}\nstderr:\n{}",

crates/cargo-anvil/tests/recipe_contracts.rs:770

  • This assertion message says "private" but the filtered-out case is publish = false / non-publishable libraries. Using consistent terminology ("non-publishable") makes failures clearer and avoids confusion with private registries.
        "private semver filtering failed\nstdout:\n{}\nstderr:\n{}",

Comment thread justfiles/anvil/checks/semver-check.just Outdated
Comment thread crates/cargo-anvil/templates/justfiles/anvil/checks/semver-check.just Outdated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4112f005-775f-4c4b-88ed-db8f0c5d1cf1
Copilot AI review requested due to automatic review settings August 28, 2026 08:27

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 22 changed files in this pull request and generated 1 comment.

Comment thread crates/cargo-anvil/docs/design/local.md Outdated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4112f005-775f-4c4b-88ed-db8f0c5d1cf1
Copilot AI review requested due to automatic review settings August 28, 2026 08:34

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 27 changed files in this pull request and generated 1 comment.

Comment thread crates/cargo-anvil/tests/recipe_contracts.rs
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4112f005-775f-4c4b-88ed-db8f0c5d1cf1
Copilot AI review requested due to automatic review settings August 31, 2026 14:16

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 23 out of 24 changed files in this pull request and generated 1 comment.

Comment thread Cargo.toml
Make dry-run project and compare the complete lockfile state, and run the dogfood regeneration gate on every pull request.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4112f005-775f-4c4b-88ed-db8f0c5d1cf1
Copilot AI review requested due to automatic review settings August 31, 2026 15:36
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4112f005-775f-4c4b-88ed-db8f0c5d1cf1

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 29 out of 30 changed files in this pull request and generated no new comments.

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

crates/cargo-anvil/src/plan.rs:391

  • This rustdoc comment references a design doc section number ("§5"), but repo guidelines prohibit citing design sections from code/rustdoc because it couples the code to the doc’s structure. Please remove the section-number reference and link to the doc generally.
    ///   subsequent runs see the divergence as resolved
    ///   (`LeaveAlone`) until the template moves again — see
    ///   [`updates.md §5`](../../docs/design/updates.md).

Copilot AI review requested due to automatic review settings August 31, 2026 15:40

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 29 out of 30 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

crates/cargo-anvil/src/plan.rs:391

  • Repo guidelines (AGENTS.md) discourage referencing design-doc section numbers from rustdoc/code comments. This link text includes a §5 section reference; please remove the section number to avoid coupling the code to the doc’s structure.
    ///   [`updates.md §5`](../../docs/design/updates.md).

Comment thread crates/cargo-gamma-lib/src/exec/verdict/hubs.rs
Comment thread crates/cargo-anvil/src/run.rs Outdated
Preserve the container execution refactor and path-containment checks while retaining dry-run manifest projection. Treat CRLF and LF lockfiles equivalently without relaxing canonical content checks.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4112f005-775f-4c4b-88ed-db8f0c5d1cf1
Copilot AI review requested due to automatic review settings August 31, 2026 17:46

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 31 changed files in this pull request and generated 1 comment.

Comment thread crates/cargo-anvil/src/plan.rs Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Copilot speaking]

Published 11 findings. 5 findings follow up on existing discussion threads.

See diagnostics
Diagnostic Value
Cache Miss

Comment thread crates/cargo-anvil/tests/recipe_contracts.rs
Comment thread crates/cargo-anvil/tests/recipe_contracts.rs
Comment thread crates/cargo-anvil/tests/recipe_contracts.rs Outdated
Comment thread crates/cargo-anvil/docs/design/checks.md Outdated
Comment thread crates/cargo-gamma-lib/src/exec/verdict/hubs.rs Outdated
Comment thread crates/cargo-anvil/docs/design/local.md Outdated
Comment thread crates/cargo-anvil/docs/design/local.md Outdated
Comment thread crates/cargo-anvil/templates/justfiles/anvil/checks/semver-check.just Outdated
Comment thread crates/cargo-anvil/templates/justfiles/anvil/helpers.just Outdated
Clarify modified-tier ownership, SemVer publishability, and Loom model invariants. Cover named-registry SemVer candidates and keep generated recipes and snapshots synchronized.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4112f005-775f-4c4b-88ed-db8f0c5d1cf1
Copilot AI review requested due to automatic review settings September 1, 2026 07:22

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 32 out of 33 changed files in this pull request and generated 2 comments.

Comment thread crates/cargo-anvil/tests/recipe_contracts.rs
Comment thread crates/cargo-anvil/src/run.rs
@martin-kolinek
martin-kolinek enabled auto-merge (squash) September 1, 2026 08:58
@martin-kolinek
martin-kolinek merged commit 6d99b19 into main Sep 1, 2026
46 checks passed
@martin-kolinek
martin-kolinek deleted the u/makolnek/anvil-generic-check-fixes branch September 1, 2026 09:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants