feat(l1): generate execution witnesses during payload execution #5708

avilagaston9 · 2025-12-22T22:03:08Z

Motivation

To provide a quicker response to debug_executionWitness requests.

Description

This PR generates execution witnesses during payload execution and stores them for a period of time (128 blocks).

Adds the node flag generate-witness.
After successfully executing a payload received via the newPayload message, if the generate-witness flag is enabled, the node will generate and store the ExecutionWitness for that block.
Upon receiving a debug_executionWitness request, the node first checks if it has the witness in storage and only generates it as a fallback.

Measurements

execution_witness_measurements.md contains a comparative analysis of debug_executionWitness latency for pre-generated vs on-demand execution witnesses.

Checklist

Updated STORE_SCHEMA_VERSION (crates/storage/lib.rs) if the PR includes breaking changes to the Store requiring a re-sync.

github-actions · 2025-12-22T22:12:44Z

Lines of code report

Total lines added: 355
Total lines removed: 0
Total lines changed: 355

Detailed view

+---------------------------------------------------------+-------+------+
| File                                                    | Lines | Diff |
+---------------------------------------------------------+-------+------+
| ethrex/cmd/ethrex/cli.rs                                | 820   | +9   |
+---------------------------------------------------------+-------+------+
| ethrex/cmd/ethrex/initializers.rs                       | 499   | +1   |
+---------------------------------------------------------+-------+------+
| ethrex/cmd/ethrex/l2/initializers.rs                    | 349   | +1   |
+---------------------------------------------------------+-------+------+
| ethrex/crates/blockchain/blockchain.rs                  | 1815  | +231 |
+---------------------------------------------------------+-------+------+
| ethrex/crates/networking/rpc/debug/execution_witness.rs | 234   | +14  |
+---------------------------------------------------------+-------+------+
| ethrex/crates/storage/api/tables.rs                     | 23    | +2   |
+---------------------------------------------------------+-------+------+
| ethrex/crates/storage/store.rs                          | 2686  | +83  |
+---------------------------------------------------------+-------+------+
| ethrex/crates/vm/levm/src/db/gen_db.rs                  | 412   | +11  |
+---------------------------------------------------------+-------+------+
| ethrex/crates/vm/levm/src/hooks/l2_hook.rs              | 568   | +3   |
+---------------------------------------------------------+-------+------+

Copilot

Pull request overview

This PR implements caching for execution witnesses to improve response times for debug_executionWitness requests. The implementation stores the last 128 execution witnesses in the database after successful block execution via the Engine API.

Key changes:

Adds a --generate-witness CLI flag to enable witness generation and caching
Stores execution witnesses after newPayload execution when the flag is enabled
Implements cache lookup in debug_executionWitness as a fast path before generating witnesses on-demand

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
docs/CLI.md	Documents the new `--generate-witness` flag and fixes whitespace formatting
crates/storage/api/tables.rs	Adds EXECUTION_WITNESSES table definition to store cached witnesses
crates/storage/store.rs	Implements witness storage, retrieval, and cleanup logic with MAX_WITNESSES=128 limit
crates/networking/rpc/rpc.rs	Adds generate_witness field to RpcApiContext to propagate the flag
crates/networking/rpc/engine/payload.rs	Generates and stores witness after successful payload execution
crates/networking/rpc/debug/execution_witness.rs	Checks cache before generating witness for single-block requests
crates/networking/rpc/test_utils.rs	Adds generate_witness=false to test contexts
crates/l2/networking/rpc/rpc.rs	Propagates generate_witness parameter in L2 RPC initialization
cmd/ethrex/l2/initializers.rs	Passes generate_witness flag to L2 RPC API
cmd/ethrex/initializers.rs	Passes generate_witness flag to L1 RPC API
cmd/ethrex/cli.rs	Defines --generate-witness CLI argument with default false

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

crates/storage/store.rs

crates/storage/api/tables.rs

crates/storage/store.rs

crates/networking/rpc/engine/payload.rs

github-actions · 2026-01-07T19:16:32Z

Benchmark Results Comparison

Benchmark Results: MstoreBench

Command	Mean [s]	Min [s]	Max [s]	Relative
`main_revm_MstoreBench`	303.2 ± 74.1	266.3	512.3	1.23 ± 0.30
`main_levm_MstoreBench`	246.7 ± 2.1	243.7	250.9	1.00 ± 0.01
`pr_revm_MstoreBench`	272.7 ± 4.4	267.3	284.1	1.11 ± 0.02

Detailed Results

Benchmark Results: BubbleSort

Command	Mean [s]	Min [s]	Max [s]	Relative
`main_revm_BubbleSort`	3.023 ± 0.018	2.999	3.056	1.00
`main_levm_BubbleSort`	3.129 ± 0.037	3.065	3.175	1.03 ± 0.01
`pr_revm_BubbleSort`	3.064 ± 0.035	3.023	3.139	1.01 ± 0.01
`pr_levm_BubbleSort`	3.182 ± 0.032	3.140	3.233	1.05 ± 0.01

Benchmark Results: ERC20Approval

Command	Mean [s]	Min [s]	Max [s]	Relative
`main_revm_ERC20Approval`	1.005 ± 0.014	0.991	1.032	1.00
`main_levm_ERC20Approval`	1.130 ± 0.012	1.118	1.160	1.12 ± 0.02
`pr_revm_ERC20Approval`	1.012 ± 0.008	0.998	1.024	1.01 ± 0.02
`pr_levm_ERC20Approval`	1.142 ± 0.018	1.118	1.186	1.14 ± 0.02

Benchmark Results: ERC20Mint

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_ERC20Mint`	138.2 ± 2.1	135.2	142.4	1.01 ± 0.02
`main_levm_ERC20Mint`	169.9 ± 4.4	165.6	180.6	1.24 ± 0.04
`pr_revm_ERC20Mint`	137.0 ± 1.7	135.1	140.4	1.00
`pr_levm_ERC20Mint`	170.1 ± 3.4	166.5	175.1	1.24 ± 0.03

Benchmark Results: ERC20Transfer

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_ERC20Transfer`	237.0 ± 2.2	233.4	239.1	1.00
`main_levm_ERC20Transfer`	284.5 ± 6.0	277.1	296.6	1.20 ± 0.03
`pr_revm_ERC20Transfer`	239.8 ± 4.0	230.8	245.8	1.01 ± 0.02
`pr_levm_ERC20Transfer`	286.4 ± 2.6	281.7	289.5	1.21 ± 0.02

Benchmark Results: Factorial

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_Factorial`	229.2 ± 2.0	226.0	232.4	1.00
`main_levm_Factorial`	275.9 ± 2.3	273.3	281.1	1.20 ± 0.01
`pr_revm_Factorial`	234.4 ± 12.4	228.0	269.4	1.02 ± 0.05
`pr_levm_Factorial`	276.4 ± 12.5	269.2	311.4	1.21 ± 0.06

Benchmark Results: FactorialRecursive

Command	Mean [s]	Min [s]	Max [s]	Relative
`main_revm_FactorialRecursive`	1.740 ± 0.024	1.709	1.788	1.00
`main_levm_FactorialRecursive`	9.008 ± 0.041	8.920	9.061	5.18 ± 0.08
`pr_revm_FactorialRecursive`	1.747 ± 0.062	1.598	1.824	1.00 ± 0.04
`pr_levm_FactorialRecursive`	9.014 ± 0.089	8.910	9.132	5.18 ± 0.09

Benchmark Results: Fibonacci

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_Fibonacci`	210.5 ± 2.2	208.5	214.7	1.00
`main_levm_Fibonacci`	272.3 ± 11.9	266.7	306.0	1.29 ± 0.06
`pr_revm_Fibonacci`	210.8 ± 2.0	208.1	213.9	1.00 ± 0.01
`pr_levm_Fibonacci`	269.8 ± 14.8	255.0	309.9	1.28 ± 0.07

Benchmark Results: FibonacciRecursive

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_FibonacciRecursive`	922.0 ± 9.7	906.7	935.0	1.20 ± 0.03
`main_levm_FibonacciRecursive`	774.5 ± 11.7	763.9	800.1	1.01 ± 0.03
`pr_revm_FibonacciRecursive`	907.3 ± 13.5	883.6	925.5	1.18 ± 0.03
`pr_levm_FibonacciRecursive`	767.5 ± 17.5	723.5	780.8	1.00

Benchmark Results: ManyHashes

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_ManyHashes`	9.1 ± 0.1	8.8	9.3	1.00
`main_levm_ManyHashes`	10.3 ± 0.1	10.2	10.4	1.13 ± 0.02
`pr_revm_ManyHashes`	9.2 ± 0.1	9.1	9.6	1.02 ± 0.02
`pr_levm_ManyHashes`	10.3 ± 0.1	10.2	10.5	1.13 ± 0.02

Benchmark Results: MstoreBench

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_MstoreBench`	303.2 ± 74.1	266.3	512.3	1.23 ± 0.30
`main_levm_MstoreBench`	246.7 ± 2.1	243.7	250.9	1.00 ± 0.01
`pr_revm_MstoreBench`	272.7 ± 4.4	267.3	284.1	1.11 ± 0.02
`pr_levm_MstoreBench`	246.3 ± 2.4	244.3	251.2	1.00

Benchmark Results: Push

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_Push`	297.4 ± 5.4	292.7	312.0	1.00
`main_levm_Push`	314.7 ± 2.7	310.0	319.3	1.06 ± 0.02
`pr_revm_Push`	300.6 ± 4.3	295.3	308.2	1.01 ± 0.02
`pr_levm_Push`	316.5 ± 3.3	311.6	323.4	1.06 ± 0.02

Benchmark Results: SstoreBench_no_opt

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_SstoreBench_no_opt`	175.4 ± 2.3	172.3	179.3	1.92 ± 0.04
`main_levm_SstoreBench_no_opt`	91.1 ± 1.7	89.4	94.0	1.00
`pr_revm_SstoreBench_no_opt`	182.7 ± 7.9	175.0	197.6	2.00 ± 0.09
`pr_levm_SstoreBench_no_opt`	92.4 ± 2.1	90.2	97.4	1.01 ± 0.03

crates/networking/rpc/debug/execution_witness.rs

iovoid · 2026-01-07T19:52:12Z

crates/storage/store.rs

+
+        let threshold = latest_block_number - MAX_WITNESSES;
+
+        self.get_oldest_witness_number()?


This could be an if-let

done 263e0e5!

crates/blockchain/blockchain.rs

JereSalo

We should specify in the CLI flag more accurately that its purpose is to generate witnesses for blocks after being synced with the network and to also cache them.
suggested help = "After syncing blocks from the network, generate execution witnesses for the synced blocks and store them in the local witness cache."
A better name for the flag in my opinion would be cache-witnesses instead.
generate_witness_from_account_updates and generate_witness_for_blocks_with_fee_configs are very untidy. We should at least open an issue because they are a pain to read IMO and they can be improved.

I have yet to review other things but I will submit this review for now.

JereSalo · 2026-01-08T20:55:47Z

crates/blockchain/blockchain.rs


-        let vm_db = StoreVmDatabase::new(self.storage.clone(), parent_header.clone())?;
-        let vm = self.new_evm(vm_db)?;
+        let (mut vm, logger) = if self.options.generate_witness && self.is_synced() {


I'd add a quick comment here saying what the purpose of this code is. Like just to execute with the database logger in order to generate the witness in one execution instead of executing more than once for this goal.

Done aa754b7!

JereSalo · 2026-01-08T21:20:23Z

crates/storage/api/tables.rs

+/// Oldest witness block column family: [`Vec<u8>`] => [`Vec<u8>`]
+/// - [`Vec<u8>`] = `b"oldest_witness_block"`
+/// - [`Vec<u8>`] = `oldest_block_number.to_le_bytes()`
+pub const OLDEST_WITNESS_BLOCK: &str = "oldest_witness_block";


The name is okay but it'd be good to specify that it's a number, like OLDEST_WITNESS_BLOCK_NUMBER

Done 14173ad!

JereSalo · 2026-01-08T21:26:03Z

crates/vm/levm/src/db/gen_db.rs

    pub current_accounts_state: CacheDB,
    pub initial_accounts_state: CacheDB,
+    pub previous_tx_accounts_state: CacheDB,


We should add comments to each because it may be confusing

Done aa754b7!

crates/blockchain/blockchain.rs

crates/vm/levm/src/db/gen_db.rs

JereSalo · 2026-01-12T14:05:09Z

crates/vm/levm/src/db/gen_db.rs

+    // Holds the initial state of accounts before the block execution started, used for calculating state transitions
    pub initial_accounts_state: CacheDB,


Maybe we should add that it could be either before a block or before a batch of blocks. This is because if you have a batch of 100 blocks the initial_accounts_state is going to store the initial state before those 100 blocks. Current explanation may be leaving that case out

JereSalo · 2026-01-12T14:08:32Z

crates/vm/levm/src/db/gen_db.rs

    pub current_accounts_state: CacheDB,
+    // Holds the initial state of accounts before the block execution started, used for calculating state transitions
    pub initial_accounts_state: CacheDB,
+    // Holds the state of accounts before the current transaction started, used for calculating state transitions per transaction (e.g. pipeline)


I'd be more specific in the parenthesis. Saying something like (used in pipeline execution) would be best. Just for people that are new to this.

JereSalo

Most important thing is to improve code docs in GeneralizedDatabase attributes.
I think that after addressing all comments it'll be ready to merge

feat(l1): cache execution witnesses

6e55b7b

github-actions bot assigned avilagaston9 Dec 22, 2025

github-actions bot added the L1 Ethereum client label Dec 22, 2025

github-project-automation bot added this to ethrex_l1 Dec 22, 2025

Fix linter

b855dd0

avilagaston9 added 3 commits December 23, 2025 10:50

Update cli docs

994b485

fix: return rpc witness

613ea65

Remove unneeded check

3302c1b

avilagaston9 marked this pull request as ready for review December 23, 2025 21:17

avilagaston9 requested a review from ilitteri as a code owner December 23, 2025 21:17

Copilot AI review requested due to automatic review settings December 23, 2025 21:17

avilagaston9 requested review from a team and ManuelBilbao as code owners December 23, 2025 21:17

ethrex-project-sync bot moved this to In Review in ethrex_l1 Dec 23, 2025

Copilot started reviewing on behalf of avilagaston9 December 23, 2025 21:18 View session

Copilot AI reviewed Dec 23, 2025

View reviewed changes

avilagaston9 added 4 commits December 23, 2025 18:24

Address comments

0b0e77e

don't return an error if witness generation fails

b17523a

Update table docs

0a3676f

Avoid returning errors

540c678

tomip01 approved these changes Jan 5, 2026

View reviewed changes

crates/networking/rpc/engine/payload.rs Outdated Show resolved Hide resolved

crates/networking/rpc/engine/payload.rs Outdated Show resolved Hide resolved

avilagaston9 added 5 commits January 5, 2026 10:28

Address comments

cd2d029

refactor: avoid re-execution

54dbcfc

Remove comments

fa11849

Only generate witness when synced

d207b46

fix: keep initial state unchanged

d1461e5

iovoid reviewed Jan 7, 2026

View reviewed changes

fix: use iterator properly

263e0e5

Add measurements doc

73269b7

JereSalo self-requested a review January 8, 2026 20:09

avilagaston9 changed the title ~~feat(l1): cache execution witnesses~~ feat(l1): generate execution witnesses during payload execution Jan 8, 2026

avilagaston9 added 2 commits January 8, 2026 17:38

rename cached/non-cached to pre-generated/on-demand

2fde629

Add observation about the rpc node

0343b46

JereSalo reviewed Jan 8, 2026

View reviewed changes

Avoid storing the block in generate_witness_from_account_updates

187f934

tomip01 reviewed Jan 9, 2026

View reviewed changes

Rename oldest_witness_block

14173ad

avilagaston9 mentioned this pull request Jan 9, 2026

Refactor repeated code in witness generation #5795

Open

avilagaston9 added 5 commits January 9, 2026 11:25

Clarify code with comments

aa754b7

Refactor with let chain

b6f7f9b

Update comment

2ed6d66

Update CLI.md

6b62d25

Merge branch 'main' into feat/l1/cache_execution_witnesses

83e202a

JereSalo reviewed Jan 12, 2026

View reviewed changes

Merge branch 'main' into feat/l1/cache_execution_witnesses

b7d9862

JereSalo reviewed Jan 12, 2026

View reviewed changes


		let threshold = latest_block_number - MAX_WITNESSES;

		self.get_oldest_witness_number()?

		// Holds the initial state of accounts before the block execution started, used for calculating state transitions
		pub initial_accounts_state: CacheDB,

feat(l1): generate execution witnesses during payload execution #5708

Are you sure you want to change the base?

feat(l1): generate execution witnesses during payload execution #5708

Uh oh!

Conversation

avilagaston9 commented Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Lines of code report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmark Results Comparison

Benchmark Results: MstoreBench

Benchmark Results: BubbleSort

Benchmark Results: ERC20Approval

Benchmark Results: ERC20Mint

Benchmark Results: ERC20Transfer

Benchmark Results: Factorial

Benchmark Results: FactorialRecursive

Benchmark Results: Fibonacci

Benchmark Results: FibonacciRecursive

Benchmark Results: ManyHashes

Benchmark Results: MstoreBench

Benchmark Results: Push

Benchmark Results: SstoreBench_no_opt

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

JereSalo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JereSalo left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

avilagaston9 commented Dec 22, 2025 •

edited

Loading

github-actions bot commented Dec 22, 2025 •

edited

Loading

github-actions bot commented Jan 7, 2026 •

edited

Loading