Skip to content

Conversation

@chfast
Copy link
Member

@chfast chfast commented Jun 2, 2025

πŸ—’οΈ Description

This refactors the test_worst_modexp() with parametrized precompile inputs. It adds inputs consting ~500 gas for the EIP-7883 review.

πŸ”— Related Issues

βœ… Checklist

  • All: Set appropriate labels for the changes.
  • All: Considered squashing commits to improve commit history.
  • All: Added an entry to CHANGELOG.md.
  • All: Considered updating the online docs in the ./docs/ directory.
  • Tests: All converted JSON/YML tests from ethereum/tests/tests/static have been assigned @ported_from marker.
  • Tests: A PR with removal of converted JSON/YML blockchain tests from ethereum/tests have been opened.
  • Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
  • Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.
  • Tests: For PRs implementing a missed test case, update the post-mortem document to add an entry the list.

@chfast chfast requested a review from jsign June 2, 2025 15:37
@chfast chfast added scope:tests Scope: Changes EL client test cases in `./tests` type:refactor Type: Refactor type:feat type: Feature feature:zkevm labels Jun 2, 2025
@chfast
Copy link
Member Author

chfast commented Jun 2, 2025

Benchmark results

Geth

evm statetest --bench fixtures/state_tests/zkevm/worst_compute/worst_modexp.json
{"stateRoot": "0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b"}
{"stateRoot": "0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b"}
{"stateRoot": "0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b"}
{"stateRoot": "0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b"}
[
  {
    "name": "tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Osaka-state_test-worst_24b]",
    "pass": true,
    "stateRoot": "0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b",
    "fork": "Osaka",
    "benchStats": {
      "time": 1557245740,
      "allocs": 6412901,
      "bytesAllocated": 352983712,
      "gasUsed": 72000000
    }
  },
  {
    "name": "tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Osaka-state_test-worst_32b]",
    "pass": true,
    "stateRoot": "0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b",
    "fork": "Osaka",
    "benchStats": {
      "time": 1056881439,
      "allocs": 2783521,
      "bytesAllocated": 169499480,
      "gasUsed": 72000000
    }
  },
  {
    "name": "tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Osaka-state_test-worst_8b]",
    "pass": true,
    "stateRoot": "0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b",
    "fork": "Osaka",
    "benchStats": {
      "time": 2286681115,
      "allocs": 6556145,
      "bytesAllocated": 261662376,
      "gasUsed": 72000000
    }
  },
  {
    "name": "tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Osaka-state_test-worst_16b]",
    "pass": true,
    "stateRoot": "0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b",
    "fork": "Osaka",
    "benchStats": {
      "time": 1623244585,
      "allocs": 6907389,
      "bytesAllocated": 323491848,
      "gasUsed": 72000000
    }
  }
]

Besu

besu-evm state-test fixtures/state_tests/zkevm/worst_compute/worst_modexp.json
{"output":"","gasUsed":"0x44aa200","time":1530777000,"Mgps":"47.035","test":"tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Osaka-state_test-worst_16b]","fork":"Osaka","d":0,"g":0,"v":0,"stateRoot":"0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b","postLogsHash":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347","pass":true,"error":"Out of gas"}
{"output":"","gasUsed":"0x44aa200","time":1076826660,"Mgps":"66.863","test":"tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Osaka-state_test-worst_24b]","fork":"Osaka","d":0,"g":0,"v":0,"stateRoot":"0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b","postLogsHash":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347","pass":true,"error":"Out of gas"}
{"output":"","gasUsed":"0x44aa200","time":678852354,"Mgps":"106.061","test":"tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Osaka-state_test-worst_32b]","fork":"Osaka","d":0,"g":0,"v":0,"stateRoot":"0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b","postLogsHash":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347","pass":true,"error":"Out of gas"}
{"output":"","gasUsed":"0x44aa200","time":1807314427,"Mgps":"39.838","test":"tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Osaka-state_test-worst_8b]","fork":"Osaka","d":0,"g":0,"v":0,"stateRoot":"0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b","postLogsHash":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347","pass":true,"error":"Out of gas"}

@marioevz
Copy link
Member

marioevz commented Jun 3, 2025

This PR might be in conflict with this commit d9da7de from #1579

@chfast
Copy link
Member Author

chfast commented Jun 3, 2025

This PR might be in conflict with this commit d9da7de from #1579

I will wait.

@marioevz
Copy link
Member

marioevz commented Jun 4, 2025

#1579 is merged, can now rebase to main and check if everything is still working.

This refactors the `test_worst_modexp()` with parametrized precompile
inputs. It adds inputs consting ~500 gas for the EIP-7883 review.
@chfast chfast marked this pull request as ready for review June 4, 2025 16:40
Copy link
Collaborator

@jsign jsign left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Cancun-blockchain_test_from_state_test-mod_even_32b_exp_40]-1 22858304416
tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Cancun-blockchain_test_from_state_test-mod_even_32b_exp_96]-1 26373254772
tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Cancun-blockchain_test_from_state_test-mod_even_32b_exp_256]-1        28089552372
tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Cancun-blockchain_test_from_state_test-mod_even_24b_exp_168]-1        29398879265
tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Cancun-blockchain_test_from_state_test-mod_even_16b_exp_320]-1        36480039911
tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Cancun-blockchain_test_from_state_test-mod_even_8b_exp_896]-1 65149073029

@marioevz marioevz merged commit a13fef6 into ethereum:main Jun 18, 2025
15 checks passed
@marioevz marioevz deleted the bench/modexp branch June 18, 2025 16:28
kclowes pushed a commit to kclowes/execution-spec-tests that referenced this pull request Oct 20, 2025
* feat(tests): add more worst-case modexp benchmarks

This refactors the `test_worst_modexp()` with parametrized precompile
inputs. It adds inputs consting ~500 gas for the EIP-7883 review.

* Add more cases

---------

Co-authored-by: Mario Vega <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature:zkevm scope:tests Scope: Changes EL client test cases in `./tests` type:feat type: Feature type:refactor Type: Refactor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants