feat(l1): generate execution witnesses during payload execution #5708

avilagaston9 · 2025-12-22T22:03:08Z

Motivation

To provide a quicker response to debug_executionWitness requests.

Description

This PR generates execution witnesses during payload execution and stores them for a period of time (128 blocks).

Adds the node flag generate-witness.
After successfully executing a payload received via the newPayload message, if the generate-witness flag is enabled, the node will generate and store the ExecutionWitness for that block.
Upon receiving a debug_executionWitness request, the node first checks if it has the witness in storage and only generates it as a fallback.

Measurements

execution_witness_measurements.md contains a comparative analysis of debug_executionWitness latency for pre-generated vs on-demand execution witnesses.

Checklist

Updated STORE_SCHEMA_VERSION (crates/storage/lib.rs) if the PR includes breaking changes to the Store requiring a re-sync.

github-actions · 2025-12-22T22:12:44Z

Lines of code report

Total lines added: 382
Total lines removed: 0
Total lines changed: 382

Detailed view

+---------------------------------------------------------+-------+------+
| File                                                    | Lines | Diff |
+---------------------------------------------------------+-------+------+
| ethrex/cmd/ethrex/cli.rs                                | 820   | +9   |
+---------------------------------------------------------+-------+------+
| ethrex/cmd/ethrex/initializers.rs                       | 507   | +1   |
+---------------------------------------------------------+-------+------+
| ethrex/cmd/ethrex/l2/initializers.rs                    | 349   | +1   |
+---------------------------------------------------------+-------+------+
| ethrex/crates/blockchain/blockchain.rs                  | 1861  | +277 |
+---------------------------------------------------------+-------+------+
| ethrex/crates/networking/rpc/debug/execution_witness.rs | 234   | +14  |
+---------------------------------------------------------+-------+------+
| ethrex/crates/storage/api/tables.rs                     | 22    | +1   |
+---------------------------------------------------------+-------+------+
| ethrex/crates/storage/store.rs                          | 2684  | +79  |
+---------------------------------------------------------+-------+------+

Copilot

Pull request overview

This PR implements caching for execution witnesses to improve response times for debug_executionWitness requests. The implementation stores the last 128 execution witnesses in the database after successful block execution via the Engine API.

Key changes:

Adds a --generate-witness CLI flag to enable witness generation and caching
Stores execution witnesses after newPayload execution when the flag is enabled
Implements cache lookup in debug_executionWitness as a fast path before generating witnesses on-demand

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
docs/CLI.md	Documents the new `--generate-witness` flag and fixes whitespace formatting
crates/storage/api/tables.rs	Adds EXECUTION_WITNESSES table definition to store cached witnesses
crates/storage/store.rs	Implements witness storage, retrieval, and cleanup logic with MAX_WITNESSES=128 limit
crates/networking/rpc/rpc.rs	Adds generate_witness field to RpcApiContext to propagate the flag
crates/networking/rpc/engine/payload.rs	Generates and stores witness after successful payload execution
crates/networking/rpc/debug/execution_witness.rs	Checks cache before generating witness for single-block requests
crates/networking/rpc/test_utils.rs	Adds generate_witness=false to test contexts
crates/l2/networking/rpc/rpc.rs	Propagates generate_witness parameter in L2 RPC initialization
cmd/ethrex/l2/initializers.rs	Passes generate_witness flag to L2 RPC API
cmd/ethrex/initializers.rs	Passes generate_witness flag to L1 RPC API
cmd/ethrex/cli.rs	Defines --generate-witness CLI argument with default false

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

crates/storage/store.rs

crates/storage/api/tables.rs

crates/storage/store.rs

crates/networking/rpc/engine/payload.rs

github-actions · 2026-01-07T19:16:32Z

Benchmark Results Comparison

Benchmark Results: MstoreBench

Command	Mean [s]	Min [s]	Max [s]	Relative
`main_revm_MstoreBench`	303.2 ± 74.1	266.3	512.3	1.23 ± 0.30
`main_levm_MstoreBench`	246.7 ± 2.1	243.7	250.9	1.00 ± 0.01
`pr_revm_MstoreBench`	272.7 ± 4.4	267.3	284.1	1.11 ± 0.02

Detailed Results

Benchmark Results: BubbleSort

Command	Mean [s]	Min [s]	Max [s]	Relative
`main_revm_BubbleSort`	3.023 ± 0.018	2.999	3.056	1.00
`main_levm_BubbleSort`	3.129 ± 0.037	3.065	3.175	1.03 ± 0.01
`pr_revm_BubbleSort`	3.064 ± 0.035	3.023	3.139	1.01 ± 0.01
`pr_levm_BubbleSort`	3.182 ± 0.032	3.140	3.233	1.05 ± 0.01

Benchmark Results: ERC20Approval

Command	Mean [s]	Min [s]	Max [s]	Relative
`main_revm_ERC20Approval`	1.005 ± 0.014	0.991	1.032	1.00
`main_levm_ERC20Approval`	1.130 ± 0.012	1.118	1.160	1.12 ± 0.02
`pr_revm_ERC20Approval`	1.012 ± 0.008	0.998	1.024	1.01 ± 0.02
`pr_levm_ERC20Approval`	1.142 ± 0.018	1.118	1.186	1.14 ± 0.02

Benchmark Results: ERC20Mint

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_ERC20Mint`	138.2 ± 2.1	135.2	142.4	1.01 ± 0.02
`main_levm_ERC20Mint`	169.9 ± 4.4	165.6	180.6	1.24 ± 0.04
`pr_revm_ERC20Mint`	137.0 ± 1.7	135.1	140.4	1.00
`pr_levm_ERC20Mint`	170.1 ± 3.4	166.5	175.1	1.24 ± 0.03

Benchmark Results: ERC20Transfer

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_ERC20Transfer`	237.0 ± 2.2	233.4	239.1	1.00
`main_levm_ERC20Transfer`	284.5 ± 6.0	277.1	296.6	1.20 ± 0.03
`pr_revm_ERC20Transfer`	239.8 ± 4.0	230.8	245.8	1.01 ± 0.02
`pr_levm_ERC20Transfer`	286.4 ± 2.6	281.7	289.5	1.21 ± 0.02

Benchmark Results: Factorial

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_Factorial`	229.2 ± 2.0	226.0	232.4	1.00
`main_levm_Factorial`	275.9 ± 2.3	273.3	281.1	1.20 ± 0.01
`pr_revm_Factorial`	234.4 ± 12.4	228.0	269.4	1.02 ± 0.05
`pr_levm_Factorial`	276.4 ± 12.5	269.2	311.4	1.21 ± 0.06

Benchmark Results: FactorialRecursive

Command	Mean [s]	Min [s]	Max [s]	Relative
`main_revm_FactorialRecursive`	1.740 ± 0.024	1.709	1.788	1.00
`main_levm_FactorialRecursive`	9.008 ± 0.041	8.920	9.061	5.18 ± 0.08
`pr_revm_FactorialRecursive`	1.747 ± 0.062	1.598	1.824	1.00 ± 0.04
`pr_levm_FactorialRecursive`	9.014 ± 0.089	8.910	9.132	5.18 ± 0.09

Benchmark Results: Fibonacci

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_Fibonacci`	210.5 ± 2.2	208.5	214.7	1.00
`main_levm_Fibonacci`	272.3 ± 11.9	266.7	306.0	1.29 ± 0.06
`pr_revm_Fibonacci`	210.8 ± 2.0	208.1	213.9	1.00 ± 0.01
`pr_levm_Fibonacci`	269.8 ± 14.8	255.0	309.9	1.28 ± 0.07

Benchmark Results: FibonacciRecursive

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_FibonacciRecursive`	922.0 ± 9.7	906.7	935.0	1.20 ± 0.03
`main_levm_FibonacciRecursive`	774.5 ± 11.7	763.9	800.1	1.01 ± 0.03
`pr_revm_FibonacciRecursive`	907.3 ± 13.5	883.6	925.5	1.18 ± 0.03
`pr_levm_FibonacciRecursive`	767.5 ± 17.5	723.5	780.8	1.00

Benchmark Results: ManyHashes

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_ManyHashes`	9.1 ± 0.1	8.8	9.3	1.00
`main_levm_ManyHashes`	10.3 ± 0.1	10.2	10.4	1.13 ± 0.02
`pr_revm_ManyHashes`	9.2 ± 0.1	9.1	9.6	1.02 ± 0.02
`pr_levm_ManyHashes`	10.3 ± 0.1	10.2	10.5	1.13 ± 0.02

Benchmark Results: MstoreBench

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_MstoreBench`	303.2 ± 74.1	266.3	512.3	1.23 ± 0.30
`main_levm_MstoreBench`	246.7 ± 2.1	243.7	250.9	1.00 ± 0.01
`pr_revm_MstoreBench`	272.7 ± 4.4	267.3	284.1	1.11 ± 0.02
`pr_levm_MstoreBench`	246.3 ± 2.4	244.3	251.2	1.00

Benchmark Results: Push

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_Push`	297.4 ± 5.4	292.7	312.0	1.00
`main_levm_Push`	314.7 ± 2.7	310.0	319.3	1.06 ± 0.02
`pr_revm_Push`	300.6 ± 4.3	295.3	308.2	1.01 ± 0.02
`pr_levm_Push`	316.5 ± 3.3	311.6	323.4	1.06 ± 0.02

Benchmark Results: SstoreBench_no_opt

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_SstoreBench_no_opt`	175.4 ± 2.3	172.3	179.3	1.92 ± 0.04
`main_levm_SstoreBench_no_opt`	91.1 ± 1.7	89.4	94.0	1.00
`pr_revm_SstoreBench_no_opt`	182.7 ± 7.9	175.0	197.6	2.00 ± 0.09
`pr_levm_SstoreBench_no_opt`	92.4 ± 2.1	90.2	97.4	1.01 ± 0.03

crates/networking/rpc/debug/execution_witness.rs

iovoid · 2026-01-07T19:52:12Z

crates/storage/store.rs

+
+        let threshold = latest_block_number - MAX_WITNESSES;
+
+        self.get_oldest_witness_number()?


This could be an if-let

done 263e0e5!

I think the change got undone

Mmm, I think you are watching an outdated version

crates/blockchain/blockchain.rs

JereSalo · 2026-01-12T14:05:09Z

crates/vm/levm/src/db/gen_db.rs

+    // Holds the initial state of accounts before the block execution started, used for calculating state transitions
    pub initial_accounts_state: CacheDB,


Maybe we should add that it could be either before a block or before a batch of blocks. This is because if you have a batch of 100 blocks the initial_accounts_state is going to store the initial state before those 100 blocks. Current explanation may be leaving that case out

JereSalo · 2026-01-12T14:08:32Z

crates/vm/levm/src/db/gen_db.rs

    pub current_accounts_state: CacheDB,
+    // Holds the initial state of accounts before the block execution started, used for calculating state transitions
    pub initial_accounts_state: CacheDB,
+    // Holds the state of accounts before the current transaction started, used for calculating state transitions per transaction (e.g. pipeline)


I'd be more specific in the parenthesis. Saying something like (used in pipeline execution) would be best. Just for people that are new to this.

I changed the PR. We now accumulate AccountUpdates inside handle_merkleization() only when generate-witness is enabled. If it is not enabled, this PR has no impact.

That's good news. I'm glad you found another way

JereSalo

Most important thing is to improve code docs in GeneralizedDatabase attributes.
I think that after addressing all comments it'll be ready to merge

JereSalo · 2026-01-13T19:01:05Z

I still dislike the generate-witness name for the CLI flag.
Suggestions:

precompute-witnesses
cache-witnesses
please-precompute-and-cache-the-execution-witnesses-so-debug_executionwitness-is-super-fast

I'd honestly go with one of the first two but it's up to you.

avilagaston9 · 2026-01-13T19:59:07Z

@JereSalo changed to precompute-witnesses here!

iovoid · 2026-01-14T18:32:49Z

crates/storage/api/tables.rs

+/// Oldest witness block number column family: [`Vec<u8>`] => [`Vec<u8>`]
+/// - [`Vec<u8>`] = `b"oldest_witness_block_number"`
+/// - [`Vec<u8>`] = `oldest_block_number.to_le_bytes()`
+pub const OLDEST_WITNESS_BLOCK_NUMBER: &str = "oldest_witness_block_number";


We should reuse another cf (like MISC_VALUES) instead of having a cf that holds a single key.

feat(l1): cache execution witnesses

6e55b7b

github-actions bot assigned avilagaston9 Dec 22, 2025

github-actions bot added the L1 Ethereum client label Dec 22, 2025

github-project-automation bot added this to ethrex_l1 Dec 22, 2025

Fix linter

b855dd0

avilagaston9 added 3 commits December 23, 2025 10:50

Update cli docs

994b485

fix: return rpc witness

613ea65

Remove unneeded check

3302c1b

avilagaston9 marked this pull request as ready for review December 23, 2025 21:17

avilagaston9 requested a review from ilitteri as a code owner December 23, 2025 21:17

Copilot AI review requested due to automatic review settings December 23, 2025 21:17

avilagaston9 requested review from a team and ManuelBilbao as code owners December 23, 2025 21:17

ethrex-project-sync bot moved this to In Review in ethrex_l1 Dec 23, 2025

Copilot started reviewing on behalf of avilagaston9 December 23, 2025 21:18 View session

Copilot AI reviewed Dec 23, 2025

View reviewed changes

avilagaston9 added 4 commits December 23, 2025 18:24

Address comments

0b0e77e

don't return an error if witness generation fails

b17523a

Update table docs

0a3676f

Avoid returning errors

540c678

tomip01 approved these changes Jan 5, 2026

View reviewed changes

crates/networking/rpc/engine/payload.rs Outdated Show resolved Hide resolved

crates/networking/rpc/engine/payload.rs Outdated Show resolved Hide resolved

avilagaston9 added 5 commits January 5, 2026 10:28

Address comments

cd2d029

refactor: avoid re-execution

54dbcfc

Remove comments

fa11849

Only generate witness when synced

d207b46

fix: keep initial state unchanged

d1461e5

iovoid reviewed Jan 7, 2026

View reviewed changes

fix: use iterator properly

263e0e5

avilagaston9 added 5 commits January 9, 2026 11:25

Clarify code with comments

aa754b7

Refactor with let chain

b6f7f9b

Update comment

2ed6d66

Update CLI.md

6b62d25

Merge branch 'main' into feat/l1/cache_execution_witnesses

83e202a

JereSalo reviewed Jan 12, 2026

View reviewed changes

Merge branch 'main' into feat/l1/cache_execution_witnesses

b7d9862

JereSalo reviewed Jan 12, 2026

View reviewed changes

avilagaston9 and others added 5 commits January 13, 2026 13:25

Improve doc comments

bf07cb5

refactor: accumulate account updates during merkleization

2c668de

Merge branch 'main' into feat/l1/cache_execution_witnesses

9dae85e

Fix typo

0577cc7

Fix clippy

2455d27

Change flag to precompute-witnesses

127494f

JereSalo approved these changes Jan 13, 2026

View reviewed changes

iovoid reviewed Jan 14, 2026

View reviewed changes

Reuse MISC_VALUES table

77c5354

iovoid approved these changes Jan 14, 2026

View reviewed changes

avilagaston9 and others added 2 commits January 14, 2026 16:41

Fix formatting

2f26467

Merge branch 'main' into feat/l1/cache_execution_witnesses

52782cd

ilitteri approved these changes Jan 14, 2026

View reviewed changes

ilitteri added this pull request to the merge queue Jan 14, 2026

Merged via the queue into main with commit 625da19 Jan 14, 2026
57 checks passed

ilitteri deleted the feat/l1/cache_execution_witnesses branch January 14, 2026 23:02

github-project-automation bot moved this from In Review to Done in ethrex_l1 Jan 14, 2026

Itodo-S mentioned this pull request Jan 19, 2026

refactor(l1,l2): extract duplicate witness generation logic to helper #5905

Open


		let threshold = latest_block_number - MAX_WITNESSES;

		self.get_oldest_witness_number()?

		// Holds the initial state of accounts before the block execution started, used for calculating state transitions
		pub initial_accounts_state: CacheDB,

feat(l1): generate execution witnesses during payload execution #5708

feat(l1): generate execution witnesses during payload execution #5708

Uh oh!

Conversation

avilagaston9 commented Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Lines of code report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmark Results Comparison

Benchmark Results: MstoreBench

Benchmark Results: BubbleSort

Benchmark Results: ERC20Approval

Benchmark Results: ERC20Mint

Benchmark Results: ERC20Transfer

Benchmark Results: Factorial

Benchmark Results: FactorialRecursive

Benchmark Results: Fibonacci

Benchmark Results: FibonacciRecursive

Benchmark Results: ManyHashes

Benchmark Results: MstoreBench

Benchmark Results: Push

Benchmark Results: SstoreBench_no_opt

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JereSalo left a comment

Choose a reason for hiding this comment

Uh oh!

JereSalo commented Jan 13, 2026

Uh oh!

avilagaston9 commented Jan 13, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

avilagaston9 commented Dec 22, 2025 •

edited

Loading

github-actions bot commented Dec 22, 2025 •

edited

Loading

github-actions bot commented Jan 7, 2026 •

edited

Loading