refactor(levm): rewrite of state EF tests runner first iteration

[sofiazcoaga]

Motivation

Related issue: #3496.

The idea is to incrementally develop a new EF Test runner (for state tests) that can eventually replace the current one. The main goal of the new runner is to be easy to understand and as straightforward as possible, also making it possible to easily add any new requirement.

Considerations

This first iteration was developed using the following test files as reference:

• vectors/state_tests/prague/eip2537_bls_12_381_precompiles/bls12_g1add/gas.json
• vectors/GeneralStateTests/Cancun/stEIP4844-blobtransactions/blobhashListBounds9.json
• every test in the vectors/LegacyTests/Cancun/GeneralStateTests/Cancun/stEIP1153-transientStorage/ directory.

The main changes are:

• The new Test and TestCase structures in types.
• The runner and parser simplified flows.

Files that should not be reviewed as they are full or partial copies of the original files:

• runner_v2/deserialize.rs
• runner_v2/utils.rs

This iteration excludes report-related code, option flags and other possible test case errors to be considered that will be included later.

[github-actions]

Lines of code report

Total lines added: 1090
Total lines removed: 0
Total lines changed: 1090

Detailed view

+-----------------------------------------------------+-------+------+
| File                                                | Lines | Diff |
+-----------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state/lib.rs                    | 7     | +1   |
+-----------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state/runner_v2/deserialize.rs  | 294   | +294 |
+-----------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state/runner_v2/error.rs        | 23    | +23  |
+-----------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state/runner_v2/mod.rs          | 7     | +7   |
+-----------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state/runner_v2/parser.rs       | 34    | +34  |
+-----------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state/runner_v2/result_check.rs | 168   | +168 |
+-----------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state/runner_v2/run.rs          | 11    | +11  |
+-----------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state/runner_v2/runner.rs       | 103   | +103 |
+-----------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state/runner_v2/types.rs        | 406   | +406 |
+-----------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state/runner_v2/utils.rs        | 43    | +43  |
+-----------------------------------------------------+-------+------+

[cdiielsi]

Really nice work! I like how you distributed the modules, having runner separated from result_check makes things feel less crowded and easier to navigate. I made a couple of comments, one you can consider moving on (it's about writing reports) the rest are rather subjective and totally skippable.

[cdiielsi]

Maybe the runner errors can be distributed at lest in two categories, each with its own enum, so you can distinguish between parsing errors and LEVM errors. I'm not sure about this though because I also think that it could escalate so you end up having too many nested enums and to me that also overcomplicates debugging... Maybe pin this though and consider it as you continue with the runner.

[sofiazcoaga]

I think this might be a good suggestion, for now I think it could be considered as out of scope as the new runner is still in a quite rustic state, but I will keep in mind for later iterations to improve the Errors.

[cdiielsi]

Maybe it could be useful to have the chance to write only the failed tests in the report. I think this could be regulated with a flag you set when running the tests. That being said, I like the idea of having both the succesful and failed tests on the report so you can have a better overview of what is working and what isn't. When you run all tests it's probably too much, but if you are running per directory or per tests it could be useful.

[sofiazcoaga]

I think this is a good idea that can be done when I get to writing the execution flags parts. Thank you!

[JereSalo]

Suggestion, leave in PR description the necessary steps for running the tests

[JereSalo]

Parsing test files...
Error: FailedToParseTestFile("./vectors/LegacyTests/Cancun/GeneralStateTests/Cancun/stEIP1153-transientStorage/21_tstoreCannotBeDosdOOO.json", "Failed to deserialize post field in test 21_tstoreCannotBeDosdOOO. Serde error: missing field state")

[JereSalo]

Now that each test is one struct we can filter out repeated tests, a problem in the other runner that we have was that we were running some tests twice because some files were duplicated