Skip to content

feat(levm): pectra-devnet6 eftests #1877

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
fkrause98 merged 11 commits into from
Mar 17, 2025
Merged

feat(levm): pectra-devnet6 eftests #1877

fkrause98 merged 11 commits into from
Mar 17, 2025

Conversation

@fborello-lambda
Copy link
Contributor

@fborello-lambda fborello-lambda commented Feb 5, 2025 •
edited by tomip01
Loading

Motivation

The latest test for pectra has been released. Pectra Devnet 6

Description

@fborello-lambda fborello-lambda added the levm Lambda EVM implementation label Feb 5, 2025
@fborello-lambda fborello-lambda self-assigned this Feb 5, 2025
@github-actions
Copy link

github-actions bot commented Feb 5, 2025 •
edited
Loading

Lines of code report

Total lines added: 8
Total lines removed: 28
Total lines changed: 36

Detailed view 
+----------------------------------------------------------+-------+------+
| File                                                     | Lines | Diff |
+----------------------------------------------------------+-------+------+
| ethrex/crates/vm/levm/src/hooks/default_hook.rs          | 310   | +8   |
+----------------------------------------------------------+-------+------+
| ethrex/crates/vm/levm/src/opcode_handlers/environment.rs | 359   | -16  |
+----------------------------------------------------------+-------+------+
| ethrex/crates/vm/levm/src/vm.rs                          | 418   | -12  |
+----------------------------------------------------------+-------+------+

@github-actions
Copy link

github-actions bot commented Feb 5, 2025 •
edited
Loading

Benchmark Results Comparison

PR Results

Benchmark Results: Factorial

Command Mean [ms] Min [ms] Max [ms] Relative
revm_Factorial 235.4 ± 0.6 234.4 236.3 1.00
levm_Factorial 913.0 ± 12.3 903.0 946.7 3.88 ± 0.05

Benchmark Results: Factorial - Recursive

Command Mean [s] Min [s] Max [s] Relative
revm_FactorialRecursive 1.405 ± 0.084 1.301 1.556 1.00
levm_FactorialRecursive 15.678 ± 0.141 15.515 15.953 11.16 ± 0.68

Benchmark Results: Fibonacci

Command Mean [ms] Min [ms] Max [ms] Relative
revm_Fibonacci 206.3 ± 1.1 205.1 208.1 1.00
levm_Fibonacci 907.0 ± 12.4 893.3 929.8 4.40 ± 0.06

Benchmark Results: ManyHashes

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ManyHashes 8.6 ± 0.1 8.5 8.7 1.00
levm_ManyHashes 18.4 ± 0.3 18.2 19.1 2.15 ± 0.03

Benchmark Results: BubbleSort

Command Mean [s] Min [s] Max [s] Relative
revm_BubbleSort 3.202 ± 0.025 3.170 3.240 1.00
levm_BubbleSort 6.101 ± 0.044 6.056 6.182 1.91 ± 0.02

Benchmark Results: ERC20 - Transfer

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ERC20Transfer 246.4 ± 3.6 242.1 253.9 1.00
levm_ERC20Transfer 542.1 ± 3.7 539.4 551.7 2.20 ± 0.04

Benchmark Results: ERC20 - Mint

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ERC20Mint 139.6 ± 2.2 138.1 145.6 1.00
levm_ERC20Mint 394.1 ± 34.0 354.6 451.4 2.82 ± 0.25

Benchmark Results: ERC20 - Approval

Command Mean [s] Min [s] Max [s] Relative
revm_ERC20Approval 1.020 ± 0.004 1.015 1.029 1.00
levm_ERC20Approval 2.020 ± 0.020 1.999 2.054 1.98 ± 0.02

Main Results

Benchmark Results: Factorial

Command Mean [ms] Min [ms] Max [ms] Relative
revm_Factorial 236.6 ± 2.5 234.3 240.4 1.00
levm_Factorial 906.8 ± 7.5 899.5 920.8 3.83 ± 0.05

Benchmark Results: Factorial - Recursive

Command Mean [s] Min [s] Max [s] Relative
revm_FactorialRecursive 1.417 ± 0.095 1.315 1.571 1.00
levm_FactorialRecursive 15.614 ± 0.109 15.440 15.718 11.02 ± 0.74

Benchmark Results: Fibonacci

Command Mean [ms] Min [ms] Max [ms] Relative
revm_Fibonacci 203.2 ± 3.4 201.6 212.7 1.00
levm_Fibonacci 913.3 ± 13.2 891.6 932.8 4.50 ± 0.10

Benchmark Results: ManyHashes

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ManyHashes 8.6 ± 0.1 8.6 8.7 1.00
levm_ManyHashes 18.3 ± 0.1 18.0 18.4 2.12 ± 0.02

Benchmark Results: BubbleSort

Command Mean [s] Min [s] Max [s] Relative
revm_BubbleSort 3.211 ± 0.022 3.186 3.248 1.00
levm_BubbleSort 6.087 ± 0.031 6.043 6.128 1.90 ± 0.02

Benchmark Results: ERC20 - Transfer

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ERC20Transfer 248.5 ± 4.1 244.7 257.7 1.00
levm_ERC20Transfer 539.8 ± 2.4 537.3 544.1 2.17 ± 0.04

Benchmark Results: ERC20 - Mint

Command Mean [ms] Min [ms] Max [ms] Relative
revm_ERC20Mint 138.6 ± 0.5 137.9 139.1 1.00
levm_ERC20Mint 351.7 ± 2.0 348.9 355.0 2.54 ± 0.02

Benchmark Results: ERC20 - Approval

Command Mean [s] Min [s] Max [s] Relative
revm_ERC20Approval 1.040 ± 0.010 1.029 1.060 1.00
levm_ERC20Approval 2.015 ± 0.015 1.989 2.038 1.94 ± 0.02

@fborello-lambda fborello-lambda mentioned this pull request Feb 11, 2025
Merged
@github-actions
Copy link

github-actions bot commented Feb 20, 2025 •
edited
Loading

EF Tests Comparison

Test Name MAIN PR DIFF
Summary: 16374/16773 (97.62%) 19689/20129 (97.81%) ⬆️ 3315
Prague: 2373/2373 (100.00%) 2574/2574 (100.00%) ➖️ 201
Cancun: 3579/3579 (100.00%) 4983/4983 (100.00%) ➖️ 1404
Shanghai: 221/221 (100.00%) 614/614 (100.00%) ➖️ 393
Paris: 62/62 (100.00%) 285/285 (100.00%) ➖️ 223
London: 39/39 (100.00%) 247/247 (100.00%) ➖️ 208
Berlin: 35/35 (100.00%) 205/239 (85.77%) ⬇️️ 170
Istanbul: 35/35 (100.00%) 229/230 (99.57%) ⬇️️ 194
Petersburg: 2377/2399 (99.08%) 2516/2541 (99.02%) ⬇️️ 139
Constantinople: 2338/2405 (97.21%) 2338/2405 (97.21%) ➖️
Byzantium: 2309/2330 (99.10%) 2448/2472 (99.03%) ⬇️️ 139
SpuriousDragon: 579/579 (100.00%) 579/579 (100.00%) ➖️
Tangerine: 563/650 (86.62%) 563/650 (86.62%) ➖️
Homestead: 1223/1324 (92.37%) 1346/1447 (93.02%) ⬆️ 123
Frontier: 641/742 (86.39%) 762/863 (88.30%) ⬆️ 121

@lambdaclass lambdaclass deleted a comment from github-actions bot Feb 20, 2025
@fborello-lambda fborello-lambda marked this pull request as ready for review February 20, 2025 19:50
@fborello-lambda fborello-lambda requested a review from a team as a code owner February 20, 2025 19:50
@github-actions
Copy link

github-actions bot commented Mar 11, 2025 •
edited
Loading

Benchmark Block Execution Results Comparison Against Main

Command Mean [s] Min [s] Max [s] Relative
base 141.205 ± 1.094 139.580 142.883 1.00
head 141.922 ± 0.803 140.951 143.591 1.01 ± 0.01

@tomip01 tomip01 self-assigned this Mar 11, 2025
@tomip01 tomip01 added the testing Hive tests, execution-spec-tests, etc label Mar 11, 2025
ilitteri
ilitteri reviewed Mar 14, 2025
View reviewed changes
@ilitteri
Copy link
Contributor

ilitteri commented Mar 14, 2025 •
edited
Loading

Add a new CI rule for EF Tests to be 100% from London to Prague forks

The idea is to enforce that EF tests from London to Prague should be 100% right?

fkrause98
fkrause98 reviewed Mar 14, 2025
View reviewed changes
fkrause98
fkrause98 requested changes Mar 14, 2025
View reviewed changes
@fkrause98 fkrause98 added this pull request to the merge queue Mar 17, 2025
Merged via the queue into main with commit 9f0148f Mar 17, 2025
44 checks passed
@fkrause98 fkrause98 deleted the levm/eip7702_devnet6 branch March 17, 2025 14:38
pedrobergamini pushed a commit to pedrobergamini/ethrex that referenced this pull request Aug 24, 2025
@fborello-lambda @jrchatruc @tomip01
feat(levm): pectra-devnet6 eftests (lambdaclass#1877)
c399144 
**Motivation**

The latest test for pectra has been released. [Pectra Devnet 6
](https://github.com/ethereum/execution-spec-tests/releases/tag/pectra-devnet-6%40v1.0.0)

**Description**

- Download latest tests
- Add eip7702 latest changes: ethereum/EIPs#9248
- Add a new CI rule for EF Tests to be 100% from London to Prague forks
- Fix tests from `set_code_txs_2` and `refunds` belonging to EIP 7702
and EIP 7623

---------

Co-authored-by: Javier Chatruc <[email protected]>
Co-authored-by: Tomás Paradelo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tomip01 tomip01 tomip01 left review comments

@LeanSerra LeanSerra LeanSerra approved these changes

@fkrause98 fkrause98 fkrause98 approved these changes

@ilitteri ilitteri ilitteri approved these changes

@avilagaston9 avilagaston9 avilagaston9 approved these changes

Assignees

@tomip01 tomip01

@fborello-lambda fborello-lambda

Labels

levm Lambda EVM implementation testing Hive tests, execution-spec-tests, etc

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@fborello-lambda @ilitteri @LeanSerra @fkrause98 @avilagaston9 @tomip01 @jrchatruc