chore(levm): improve organization of new levm test runner #3958

JereSalo · 2025-08-05T19:07:20Z

Motivation

this runner was inside the same folder as the other runner and also the internal structure wasn't the most organized

Description

move files
improve makefile
improve files structure

github-actions · 2025-08-05T19:12:14Z

Lines of code report

Total lines added: 1296
Total lines removed: 1
Total lines changed: 1297

Detailed view

+----------------------------------------------------------+-------+------+
| File                                                     | Lines | Diff |
+----------------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state/lib.rs                         | 6     | -1   |
+----------------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state_v2/src/lib.rs                  | 1     | +1   |
+----------------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state_v2/src/main.rs                 | 22    | +22  |
+----------------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state_v2/src/modules/deserialize.rs  | 146   | +146 |
+----------------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state_v2/src/modules/error.rs        | 8     | +8   |
+----------------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state_v2/src/modules/mod.rs          | 8     | +8   |
+----------------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state_v2/src/modules/parser.rs       | 94    | +94  |
+----------------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state_v2/src/modules/report.rs       | 133   | +133 |
+----------------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state_v2/src/modules/result_check.rs | 290   | +290 |
+----------------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state_v2/src/modules/runner.rs       | 148   | +148 |
+----------------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state_v2/src/modules/types.rs        | 408   | +408 |
+----------------------------------------------------------+-------+------+
| ethrex/cmd/ef_tests/state_v2/src/modules/utils.rs        | 38    | +38  |
+----------------------------------------------------------+-------+------+

github-actions · 2025-08-05T19:26:40Z

Benchmark Results Comparison

No significant difference was registered for any benchmark run.

Detailed Results

Benchmark Results: BubbleSort

Command	Mean [s]	Min [s]	Max [s]	Relative
`main_revm_BubbleSort`	3.144 ± 0.028	3.113	3.192	1.00
`main_levm_BubbleSort`	3.515 ± 0.021	3.492	3.560	1.12 ± 0.01
`pr_revm_BubbleSort`	3.162 ± 0.027	3.137	3.232	1.01 ± 0.01
`pr_levm_BubbleSort`	3.530 ± 0.073	3.487	3.735	1.12 ± 0.03

Benchmark Results: ERC20Approval

Command	Mean [s]	Min [s]	Max [s]	Relative
`main_revm_ERC20Approval`	1.020 ± 0.008	1.011	1.033	1.00
`main_levm_ERC20Approval`	1.200 ± 0.008	1.192	1.214	1.18 ± 0.01
`pr_revm_ERC20Approval`	1.021 ± 0.011	1.014	1.053	1.00 ± 0.01
`pr_levm_ERC20Approval`	1.194 ± 0.003	1.188	1.199	1.17 ± 0.01

Benchmark Results: ERC20Mint

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_ERC20Mint`	134.2 ± 0.9	132.9	135.6	1.00 ± 0.01
`main_levm_ERC20Mint`	162.1 ± 0.5	161.5	163.1	1.21 ± 0.01
`pr_revm_ERC20Mint`	133.9 ± 0.7	133.3	135.9	1.00
`pr_levm_ERC20Mint`	165.8 ± 0.7	164.7	166.8	1.24 ± 0.01

Benchmark Results: ERC20Transfer

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_ERC20Transfer`	239.0 ± 6.6	235.2	256.8	1.01 ± 0.03
`main_levm_ERC20Transfer`	296.4 ± 5.4	292.8	311.0	1.25 ± 0.03
`pr_revm_ERC20Transfer`	237.3 ± 3.1	234.5	245.1	1.00
`pr_levm_ERC20Transfer`	297.7 ± 5.7	293.7	313.5	1.25 ± 0.03

Benchmark Results: Factorial

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_Factorial`	229.1 ± 1.6	226.4	232.5	1.00
`main_levm_Factorial`	306.9 ± 1.2	305.6	308.7	1.34 ± 0.01
`pr_revm_Factorial`	239.5 ± 26.2	224.4	311.5	1.05 ± 0.11
`pr_levm_Factorial`	310.1 ± 5.9	306.8	326.5	1.35 ± 0.03

Benchmark Results: FactorialRecursive

Command	Mean [s]	Min [s]	Max [s]	Relative
`main_revm_FactorialRecursive`	1.552 ± 0.018	1.525	1.576	1.00
`main_levm_FactorialRecursive`	6.945 ± 0.023	6.923	6.987	4.48 ± 0.06
`pr_revm_FactorialRecursive`	1.558 ± 0.027	1.520	1.607	1.00 ± 0.02
`pr_levm_FactorialRecursive`	7.053 ± 0.020	7.023	7.081	4.55 ± 0.06

Benchmark Results: Fibonacci

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_Fibonacci`	200.8 ± 1.0	199.7	202.9	1.00 ± 0.01
`main_levm_Fibonacci`	312.7 ± 45.7	295.2	442.5	1.56 ± 0.23
`pr_revm_Fibonacci`	200.4 ± 1.0	199.1	202.2	1.00
`pr_levm_Fibonacci`	304.1 ± 12.5	296.9	338.5	1.52 ± 0.06

Benchmark Results: FibonacciRecursive

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_FibonacciRecursive`	831.0 ± 15.5	802.2	852.7	1.00
`main_levm_FibonacciRecursive`	1086.6 ± 13.0	1070.4	1115.5	1.31 ± 0.03
`pr_revm_FibonacciRecursive`	838.8 ± 12.6	819.7	852.1	1.01 ± 0.02
`pr_levm_FibonacciRecursive`	1072.3 ± 19.8	1055.9	1123.8	1.29 ± 0.03

Benchmark Results: ManyHashes

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_ManyHashes`	8.7 ± 0.1	8.6	8.8	1.00 ± 0.01
`main_levm_ManyHashes`	10.7 ± 0.3	10.5	11.4	1.24 ± 0.03
`pr_revm_ManyHashes`	8.6 ± 0.1	8.6	8.7	1.00
`pr_levm_ManyHashes`	10.6 ± 0.2	10.4	10.9	1.22 ± 0.02

Benchmark Results: MstoreBench

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_MstoreBench`	266.3 ± 3.8	263.0	274.0	1.00 ± 0.02
`main_levm_MstoreBench`	750.1 ± 3.4	747.4	759.0	2.82 ± 0.04
`pr_revm_MstoreBench`	265.9 ± 3.6	263.2	272.9	1.00
`pr_levm_MstoreBench`	751.0 ± 3.0	748.6	759.1	2.82 ± 0.04

Benchmark Results: Push

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_Push`	289.4 ± 1.0	288.1	291.2	1.00
`main_levm_Push`	882.5 ± 2.4	878.0	887.0	3.05 ± 0.01
`pr_revm_Push`	290.5 ± 3.6	287.8	300.1	1.00 ± 0.01
`pr_levm_Push`	877.8 ± 6.6	872.5	894.8	3.03 ± 0.03

Benchmark Results: SstoreBench_no_opt

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`main_revm_SstoreBench_no_opt`	157.5 ± 1.7	155.9	159.9	1.96 ± 0.03
`main_levm_SstoreBench_no_opt`	80.2 ± 1.1	78.1	82.3	1.00
`pr_revm_SstoreBench_no_opt`	158.9 ± 1.9	156.1	160.4	1.98 ± 0.04
`pr_levm_SstoreBench_no_opt`	82.4 ± 4.7	78.8	92.7	1.03 ± 0.06

Copilot

Pull Request Overview

This PR reorganizes the new LEVM runner by moving it from a subdirectory within the existing state test runner to its own separate package. The changes improve the project structure by creating a dedicated workspace member for the new runner and updating all import paths accordingly.

Extracted the new LEVM runner from cmd/ef_tests/state/runner_v2/ to its own package at cmd/ef_tests/state_v2/
Updated import paths from crate::runner_v2:: to crate::modules:: throughout the codebase
Simplified file paths for report generation by removing the runner_v2/ directory prefix

Reviewed Changes

Copilot reviewed 17 out of 22 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
cmd/ef_tests/state_v2/src/modules/*.rs	Updated import paths to use `crate::modules::` instead of `crate::runner_v2::`
cmd/ef_tests/state_v2/src/main.rs	Updated package name and import paths for the new structure
cmd/ef_tests/state_v2/src/lib.rs	New library root exposing the modules
cmd/ef_tests/state_v2/README.md	Updated documentation to reflect new package name and file paths
cmd/ef_tests/state_v2/Makefile	New Makefile with simplified test runner commands
cmd/ef_tests/state_v2/Cargo.toml	New package configuration for the extracted runner
cmd/ef_tests/state_v2/.gitignore	New gitignore for the standalone package
cmd/ef_tests/state/lib.rs	Removed reference to the old runner_v2 module
cmd/ef_tests/state/Makefile	Removed the new runner command that was delegating to the old location
cmd/ef_tests/state/Cargo.toml	Removed the test configuration for the old runner location
Cargo.toml	Added the new state_v2 package to the workspace

cmd/ef_tests/state_v2/src/modules/parser.rs

JereSalo · 2025-08-05T19:44:59Z

cmd/ef_tests/state_v2/src/modules/parser.rs

-    "dynamicAccountOverwriteEmpty_Paris.json", // Skip because it fails on REVM
-    "RevertInCreateInInitCreate2Paris.json", // Skip because it fails on REVM. See https://github.com/lambdaclass/ethrex/issues/1555
-    "RevertInCreateInInit_Paris.json", // Skip because it fails on REVM. See https://github.com/lambdaclass/ethrex/issues/1555
-    "create2collisionStorageParis.json", // Skip because it fails on REVM
-    "InitCollisionParis.json",         // Skip because it fails on REVM
-    "InitCollision.json",              // Skip because it fails on REVM


This will be tackled in next iterations, in this runner we won't use REVM for re-execution

**Motivation** - this runner was inside the same folder as the other runner and also the internal structure wasn't the most organized **Description** - move files - improve makefile - improve files structure <img width="290" height="416" alt="image" src="https://github.com/user-attachments/assets/7d4f6583-2af1-4b3f-b77b-dcf355dee897" />

…s#3958) **Motivation** - this runner was inside the same folder as the other runner and also the internal structure wasn't the most organized **Description** - move files - improve makefile - improve files structure <img width="290" height="416" alt="image" src="https://github.com/user-attachments/assets/7d4f6583-2af1-4b3f-b77b-dcf355dee897" />

move files runner

dabcc41

JereSalo self-assigned this Aug 5, 2025

JereSalo added levm Lambda EVM implementation L1 Ethereum client labels Aug 5, 2025

github-project-automation bot added this to ethrex_l1 Aug 5, 2025

ethrex-project-sync bot moved this to In Progress in ethrex_l1 Aug 5, 2025

JereSalo changed the title ~~chore(levm): move files new levm runner~~ chore(levm): improve organization of new levm runner Aug 5, 2025

JereSalo added 2 commits August 5, 2025 16:38

organize some stuff

655316f

improve makefile

9e4bcf3

JereSalo marked this pull request as ready for review August 5, 2025 19:44

Copilot AI review requested due to automatic review settings August 5, 2025 19:44

JereSalo requested a review from a team as a code owner August 5, 2025 19:44

ethrex-project-sync bot moved this from In Progress to In Review in ethrex_l1 Aug 5, 2025

Copilot AI reviewed Aug 5, 2025

View reviewed changes

cmd/ef_tests/state_v2/src/modules/parser.rs Show resolved Hide resolved

JereSalo commented Aug 5, 2025

View reviewed changes

JereSalo changed the title ~~chore(levm): improve organization of new levm runner~~ chore(levm): improve organization of new levm test runner Aug 6, 2025

change command in readme

50fb606

fmoletta approved these changes Aug 6, 2025

View reviewed changes

cdiielsi approved these changes Aug 8, 2025

View reviewed changes

JereSalo added this pull request to the merge queue Aug 8, 2025

JereSalo removed this pull request from the merge queue due to a manual request Aug 8, 2025

Merge branch 'main' into move_files_runner

8bcab12

JereSalo added this pull request to the merge queue Aug 19, 2025

Merged via the queue into main with commit 6aac0d0 Aug 19, 2025
61 of 65 checks passed

JereSalo deleted the move_files_runner branch August 19, 2025 18:27

github-project-automation bot moved this from In Review to Done in ethrex_l1 Aug 19, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

chore(levm): improve organization of new levm test runner #3958

chore(levm): improve organization of new levm test runner #3958

Uh oh!

JereSalo commented Aug 5, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Aug 5, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Aug 5, 2025 •

edited

Loading

Benchmark Results: BubbleSort

Benchmark Results: ERC20Approval

Benchmark Results: ERC20Mint

Benchmark Results: ERC20Transfer

Benchmark Results: Factorial

Benchmark Results: FactorialRecursive

Benchmark Results: Fibonacci

Benchmark Results: FibonacciRecursive

Benchmark Results: ManyHashes

Benchmark Results: MstoreBench

Benchmark Results: Push

Benchmark Results: SstoreBench_no_opt

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

JereSalo Aug 5, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chore(levm): improve organization of new levm test runner #3958

chore(levm): improve organization of new levm test runner #3958

Uh oh!

Conversation

JereSalo commented Aug 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Lines of code report

Uh oh!

github-actions bot commented Aug 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmark Results Comparison

Benchmark Results: BubbleSort

Benchmark Results: ERC20Approval

Benchmark Results: ERC20Mint

Benchmark Results: ERC20Transfer

Benchmark Results: Factorial

Benchmark Results: FactorialRecursive

Benchmark Results: Fibonacci

Benchmark Results: FibonacciRecursive

Benchmark Results: ManyHashes

Benchmark Results: MstoreBench

Benchmark Results: Push

Benchmark Results: SstoreBench_no_opt

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

JereSalo Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JereSalo commented Aug 5, 2025 •

edited

Loading

github-actions bot commented Aug 5, 2025 •

edited

Loading

github-actions bot commented Aug 5, 2025 •

edited

Loading