Skip to content

fix(cargo-anvil): make stable toolchains deterministic - #109

Merged
martin-kolinek merged 36 commits into
mainfrom
makolnek/anvil-deterministic-stable
Sep 1, 2026
Merged

fix(cargo-anvil): make stable toolchains deterministic#109
martin-kolinek merged 36 commits into
mainfrom
makolnek/anvil-deterministic-stable

Conversation

@martin-kolinek

@martin-kolinek martin-kolinek commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator

🤖 Fixes runner-dependent stable Rust selection in cargo-anvil and adds explicit MSRV testing.

What changed

  • resolve stable checks from RUSTUP_TOOLCHAIN, then a selecting root rust-toolchain/rust-toolchain.toml, then the root MSRV
  • fail instead of inheriting a runner image's moving stable toolchain
  • preserve full toolchain-file behavior, including targets, components, and profile
  • validate that MSRV fallback is uniform across workspace packages
  • run affected-package unit and integration tests under every declared root MSRV with all features and default features
  • execute MSRV tests across the same four GitHub and two ADO platform legs as ordinary PR tests so cfg-gated configurations are covered
  • allow internal builds to map the public MSRV to a provisioned toolchain through ANVIL_MSRV_TOOLCHAIN
  • apply deterministic selection before compiler/cache setup in GitHub Actions, Azure Pipelines, and container builds
  • update design docs, generated artifacts, snapshots, and resolver regression coverage

Adopter migration

Ambient runner stable is no longer accepted. Adopters must provide one of:

  • a root [workspace.package].rust-version or [package].rust-version shared by every workspace package
  • a selecting root rust-toolchain or rust-toolchain.toml
  • a preconfigured RUSTUP_TOOLCHAIN

Workspaces with missing or heterogeneous package MSRVs cannot use MSRV fallback and must select a catalog toolchain explicitly. Internal builds can override public selection with RUSTUP_TOOLCHAIN and map the public MSRV test run to a provisioned compiler with ANVIL_MSRV_TOOLCHAIN.

Motivation

Oxidizer PR #698 passed the legacy Rust 1.96.1 job but failed Anvil on a Windows ARM runner that had inherited Rust 1.98.0 and its new Clippy lints. Stable checks should not change behavior based on runner image rollout timing.

Coverage executes the ordinary test suite on the catalog nightly. The dedicated pr-msrv group runs in parallel and ensures that the same affected unit and integration tests continue to compile and pass on the minimum supported compiler across every supported host configuration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@martin-kolinek martin-kolinek changed the title Make Anvil stable toolchains deterministic fix(cargo-anvil): make stable toolchains deterministic Aug 27, 2026
@codecov-commenter

Codecov Comments Bot (codecov-commenter) commented Aug 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.7%. Comparing base (ff4aa04) to head (264c51a).

❌ Your project status has failed because the head coverage (97.7%) is below the target coverage (100.0%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
@@          Coverage Diff          @@
##            main    #109   +/-   ##
=====================================
  Coverage   97.7%   97.7%           
=====================================
  Files        286     286           
  Lines      62438   62441    +3     
=====================================
+ Hits       61009   61015    +6     
+ Misses      1429    1426    -3     
Flag Coverage Δ
linux 97.6% <100.0%> (?)
linux-arm 97.6% <100.0%> (?)
windows 97.9% <100.0%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

martin-kolinek and others added 5 commits August 27, 2026 13:50
Add a conditional Linux PR check when a selecting toolchain file differs from the workspace MSRV, with support for mapped internal toolchains.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Run affected-package tests under every declared root MSRV across the same GitHub and ADO platform matrices as ordinary PR tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Validate resolver diagnostics across stdout and stderr so the regression tests behave consistently across host PowerShell implementations.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Collapse PowerShell line wrapping before asserting resolver errors so the tests are portable across host terminal widths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assert stable resolver diagnostic fragments independently so PowerShell continuation markers do not make the tests host-dependent.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Comment thread crates/cargo-anvil/templates/anvil/resolve-stable-toolchain.ps1 Outdated
Comment thread crates/cargo-anvil/templates/anvil/resolve-stable-toolchain.ps1 Outdated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Comment thread crates/cargo-anvil/templates/anvil/resolve-stable-toolchain.ps1 Outdated
Comment thread crates/cargo-anvil/templates/anvil/resolve-stable-toolchain.ps1 Outdated
Comment thread crates/cargo-anvil/templates/anvil/resolve-stable-toolchain.ps1 Outdated
Comment thread crates/cargo-anvil/templates/justfiles/anvil/groups/pr-slow.just Outdated
Comment thread crates/cargo-anvil/docs/design/README.md
Comment thread crates/cargo-anvil/templates/justfiles/anvil/checks/msrv-test.just
martin-kolinek and others added 4 commits August 28, 2026 11:41
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Comment thread crates/cargo-anvil/templates/anvil/resolve-stable-toolchain.ps1 Outdated
Comment thread crates/cargo-anvil/templates/anvil/resolve-stable-toolchain.ps1 Outdated
Comment thread crates/cargo-anvil/templates/anvil/resolve-stable-toolchain.ps1 Outdated
martin-kolinek and others added 6 commits August 28, 2026 13:23
Remove the globally exported resolved toolchain and route stable commands through private Just helpers. Keep explicit nightly and MSRV selection at each command site, and preserve current-checkout toolchain selection for impact baselines.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Reject incomplete internal mappings, accept member MSRVs satisfied by the root floor, and cover public and mapped MSRV provisioning branches.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Install the Just bootstrap before invoking private resolver recipes in GitHub Actions, preserving recipe-based selection without assuming Just is preinstalled on hosted runners.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Accept PowerShell's platform-specific diagnostic wrapping while still asserting every actionable part of the mapped-toolchain error.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f7996e6f-5c20-4d7a-b2f7-690af95e557e
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f7996e6f-5c20-4d7a-b2f7-690af95e557e
Copilot AI review requested due to automatic review settings September 1, 2026 11:08

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 93 out of 94 changed files in this pull request and generated 2 comments.

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

justfiles/anvil/tools.just:483

  • _check-tool runs cargo {{_anvil_stable_toolchain_args}} install --list during validate-prereqs. When the stable selector falls back to +<MSRV>, this can cause rustup to auto-install the MSRV toolchain (or fail in a confusing way) just to list installed cargo tools, which makes validation non-read-only and hides the intended "install stable toolchain" guidance.

justfiles/anvil/tools.just:254

  • Test-ToolchainInstalled doesn't recognize installed stable-version toolchains when rustup normalizes them to include a patch (e.g. 1.95.0-x86_64-...). The current regex only matches ^<toolchain>(-|$), so 1.95 will not match 1.95.0-..., causing false "not installed" errors and redundant install attempts.

Comment thread justfiles/anvil/versions.just
Comment thread crates/cargo-anvil/templates/justfiles/anvil/versions.just
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f7996e6f-5c20-4d7a-b2f7-690af95e557e
Copilot AI review requested due to automatic review settings September 1, 2026 12:07

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 93 out of 94 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

justfiles/anvil/tools.just:272

  • cargo metadata writes JSON to stdout and warnings/errors to stderr. Capturing 2>&1 into $metadataText can corrupt the JSON even when cargo metadata succeeds with warnings, causing ConvertFrom-Json to throw a misleading parse error. Capture stdout only for the success path, and only merge stderr when re-running for diagnostics on failure.

Comment thread justfiles/anvil/tools.just
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f7996e6f-5c20-4d7a-b2f7-690af95e557e
Copilot AI review requested due to automatic review settings September 1, 2026 12:16
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f7996e6f-5c20-4d7a-b2f7-690af95e557e

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 93 out of 94 changed files in this pull request and generated no new comments.

Suppressed comments (4)

Previously missed (2) — in code that hasn't changed since the last review.

justfiles/anvil/tools.just:271

  • cargo metadata output is captured with 2>&1 and then piped to ConvertFrom-Json. Any warning text written to stderr (even when the command succeeds) will corrupt the JSON and make MSRV compatibility validation fail non-deterministically.

This issue also appears on line 489 of the same file.
crates/cargo-anvil/templates/justfiles/anvil/tools.just:271

  • cargo metadata output is captured with 2>&1 and then piped to ConvertFrom-Json. Any warning text written to stderr (even when the command succeeds) will corrupt the JSON and make MSRV compatibility validation fail non-deterministically.

This issue also appears on line 489 of the same file.

justfiles/anvil/tools.just:491

  • If cargo install --list fails (for example because RUSTUP_AUTO_INSTALL=0 prevents auto-provisioning of the selected toolchain), this code silently treats the tool as missing and prints an incorrect install hint. It should fail with a message that the selected stable toolchain/cargo invocation is unavailable.
    crates/cargo-anvil/templates/justfiles/anvil/tools.just:491
  • If cargo install --list fails (for example because RUSTUP_AUTO_INSTALL=0 prevents auto-provisioning of the selected toolchain), this code silently treats the tool as missing and prints an incorrect install hint. It should fail with a message that the selected stable toolchain/cargo invocation is unavailable.

Copilot AI review requested due to automatic review settings September 1, 2026 12:21

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 93 out of 94 changed files in this pull request and generated no new comments.

Suppressed comments (4)

Previously missed (2) — in code that hasn't changed since the last review.

justfiles/anvil/tools.just:633

  • When the stable selection is provided via RUSTUP_TOOLCHAIN or a repository rust-toolchain(.toml), this branch runs rustup component add <component> without --toolchain. rustup component add targets rustup's default toolchain, not the toolchain selected by RUSTUP_TOOLCHAIN or by a toolchain file, so clippy/rustfmt/etc. can be installed onto the wrong toolchain and the subsequent cargo clippy/cargo fmt checks can still fail.

This issue also appears on line 715 of the same file.
crates/cargo-anvil/templates/justfiles/anvil/tools.just:633

  • When the stable selection is provided via RUSTUP_TOOLCHAIN or a repository rust-toolchain(.toml), this branch runs rustup component add <component> without --toolchain. rustup component add targets rustup's default toolchain, not the toolchain selected by RUSTUP_TOOLCHAIN or by a toolchain file, so clippy/rustfmt/etc. can be installed onto the wrong toolchain and the subsequent cargo clippy/cargo fmt checks can still fail.

This issue also appears on line 715 of the same file.

justfiles/anvil/tools.just:719

  • The remediation hint for missing components suggests rustup component add <component> for the default toolchain when the stable selection comes from RUSTUP_TOOLCHAIN or a repository rust-toolchain(.toml). That command installs onto rustup's default toolchain, which may not be the toolchain Anvil is actually using, so the suggested fix can be ineffective.
    crates/cargo-anvil/templates/justfiles/anvil/tools.just:719
  • The remediation hint for missing components suggests rustup component add <component> for the default toolchain when the stable selection comes from RUSTUP_TOOLCHAIN or a repository rust-toolchain(.toml). That command installs onto rustup's default toolchain, which may not be the toolchain Anvil is actually using, so the suggested fix can be ineffective.

Comment thread crates/cargo-anvil/templates/justfiles/anvil/tools.just Outdated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f7996e6f-5c20-4d7a-b2f7-690af95e557e
Copilot AI review requested due to automatic review settings September 1, 2026 13:21

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 93 out of 94 changed files in this pull request and generated no new comments.

Suppressed comments (2)

justfiles/anvil/tools.just:254

  • Test-ToolchainInstalled() only matches toolchain list entries that are exactly '-' or '' at the start of the line. For MSRV values like "1.95" (this repo uses rust-version = "1.95"), rustup typically lists the installed toolchain as "1.95.0-", so this check will incorrectly report the toolchain as not installed and can break validate-workspace-msrv / re-run installs repeatedly. Consider also matching a dot after the requested prefix so "1.95" matches "1.95.0-…".
    crates/cargo-anvil/templates/justfiles/anvil/tools.just:254
  • Same as the emitted justfile: Test-ToolchainInstalled() only matches '-' or '' at line start. For MSRV values like "1.95" rustup typically lists the installed toolchain as "1.95.0-", so this check can incorrectly report the MSRV toolchain as missing and break validate-workspace-msrv / repeat installs. Match a dot as well so "1.95" matches "1.95.0-…".

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f7996e6f-5c20-4d7a-b2f7-690af95e557e
Copilot AI review requested due to automatic review settings September 1, 2026 16:28
@martin-kolinek
martin-kolinek enabled auto-merge (squash) September 1, 2026 16:31

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 93 out of 94 changed files in this pull request and generated 2 comments.

Suppressed comments (1)

justfiles/anvil/tools.just:695

  • In _check-component, when toolchain="default" and stable selection resolves to an explicit MSRV (via _anvil_stable_toolchain_args), failures in the rustc ... --print sysroot probe report toolchain 'default' not installed and suggest rustup toolchain install default, which is not actionable ("default" isn't a rustup toolchain name).

Comment thread justfiles/anvil/checks/msrv-test.just
Comment thread justfiles/anvil/tools.just
@martin-kolinek
martin-kolinek merged commit d36599e into main Sep 1, 2026
50 checks passed
@martin-kolinek
martin-kolinek deleted the makolnek/anvil-deterministic-stable branch September 1, 2026 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants