Skip to content

Conversation

Jiloc
Copy link
Contributor

@Jiloc Jiloc commented Oct 6, 2025

Description

This PR introduces an overhaul to the smart contract consensus testing suite, replacing manual test setups with a declarative framework. The goal is to make it significantly easier to write robust, multi-epoch, and multi-Clarity-version consensus tests.

We focus on ensuring we don't break consensus for state transitions that already happened, can happen now, or will happen in the future.

  • Past Events: For consensus-breaking changes in past epochs (e.g., 2.0, 3.1):

    • If a specific scenario never actually occurred on mainnet, we don't need to test it, as it can no longer happen.
    • If a scenario did happen, Genesis Sync is the mechanism responsible for capturing the real-world historical state. Our test suite does not need to replicate it.
  • Current and Future Events:

    • Contract Calls: Contracts deployed in old epochs (e.g., 2.0+) can still be called today (in 3.2+) and in the future. Therefore, the strategy here is to deploy test contracts across all epochs (2.0+) using every Clarity version that was valid at the time, but execute the function calls only from the current epoch (3.2) onwards.
    • Everything Else (Deploys without calls, Transfers, etc.): For new transactions that don't interact with historical state, we only need to test them from the current (3.2) and future epochs.
  • New Macros:

    • contract_call_consensus_test!: Allows defining a full contract deployment and function call test scenario that automatically implements the strategy described above.
    • contract_deploy_consensus_test!: A specialized version for testing only the contract deployment phase, focusing on current and future epochs.

Applicable issues

Additional info (benefits, drawbacks, caveats)

Checklist

  • Test coverage for new or modified code paths
  • Changelog is updated
  • Required documentation changes (e.g., docs/rpc/openapi.yaml and rpc-endpoints.md for v2 endpoints, event-dispatcher.md for new events)
  • New clarity functions have corresponding PR in clarity-benchmarking repo
  • New integration test(s) added to bitcoin-tests.yml

@Jiloc Jiloc self-assigned this Oct 7, 2025
@Jiloc Jiloc marked this pull request as ready for review October 7, 2025 16:53
@Jiloc Jiloc requested review from a team as code owners October 7, 2025 16:53
@Jiloc Jiloc marked this pull request as draft October 7, 2025 16:54
Copy link

codecov bot commented Oct 7, 2025

Codecov Report

❌ Patch coverage is 79.29515% with 47 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.48%. Comparing base (7762d88) to head (d7cd95e).
⚠️ Report is 22 commits behind head on develop.

Files with missing lines Patch % Lines
stackslib/src/chainstate/tests/consensus.rs 79.29% 47 Missing ⚠️

❌ Your project status has failed because the head coverage (68.48%) is below the target coverage (80.00%). You can increase the head coverage or adjust the target coverage.

❗ There is a different number of reports uploaded between BASE (7762d88) and HEAD (d7cd95e). Click for more details.

HEAD has 51 uploads less than BASE
Flag BASE (7762d88) HEAD (d7cd95e)
140 89
Additional details and impacted files
@@             Coverage Diff              @@
##           develop    #6562       +/-   ##
============================================
- Coverage    79.96%   68.48%   -11.48%     
============================================
  Files          571      571               
  Lines       351591   351613       +22     
============================================
- Hits        281135   240813    -40322     
- Misses       70456   110800    +40344     
Files with missing lines Coverage Δ
clarity/src/vm/version.rs 76.19% <ø> (+16.66%) ⬆️
stacks-common/src/types/mod.rs 75.69% <ø> (-5.38%) ⬇️
stackslib/src/chainstate/tests/consensus.rs 84.00% <79.29%> (-14.60%) ⬇️

... and 296 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7762d88...d7cd95e. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Jiloc Jiloc changed the title feat: add consensus_test macro feat: add macros for contract consensus tests Oct 10, 2025
@Jiloc Jiloc modified the milestones: 3.2.0.0.2, 3.3.0.0.0 Oct 10, 2025
@Jiloc Jiloc added the aac Avoiding Accidental Consensus label Oct 10, 2025
@Jiloc Jiloc requested a review from fdefelici October 10, 2025 14:14
@Jiloc Jiloc requested a review from fdefelici October 14, 2025 10:19
jferrant
jferrant previously approved these changes Oct 14, 2025
Copy link
Contributor

@jferrant jferrant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I think this will be a great help when we start flushing out the tests.

Copy link
Contributor

@fdefelici fdefelici left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

heads-up: the are some conflicts to manage

@Jiloc Jiloc requested review from fdefelici and jferrant October 15, 2025 11:22
@fdefelici fdefelici requested review from federico-stacks and removed request for fdefelici October 15, 2025 11:41
@github-project-automation github-project-automation bot moved this from Status: In Review to Status: 💻 In Progress in Stacks Core Eng Oct 15, 2025
@jferrant jferrant added this pull request to the merge queue Oct 15, 2025
Merged via the queue into stacks-network:develop with commit 67e6805 Oct 15, 2025
304 of 307 checks passed
@github-project-automation github-project-automation bot moved this from Status: 💻 In Progress to Status: ✅ Done in Stacks Core Eng Oct 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

aac Avoiding Accidental Consensus aac-testing Avoiding Accidental Consensus Testing Specific Task

Projects

Status: Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

AAC: add automatic permutations of the clarity/epoch versions for a test AAC: Add clarity version support to the Test harness

4 participants