Offloading KZG Proof Computation from the beacon node #7117

jimmygchen · 2025-03-12T12:30:00Z

Issue Addressed

Addresses #7108

Add EL integration for getPayloadV5 and getBlobsV2
Offload proof computation and use proofs from EL RPC APIs

Tasks

EL interop ready for peerdas-devnet-6
Update getBlobsSidecars Beacon API
Remove data column receiver and await for the computation, so that we can check data column slashing.
KZG verification during block production
Update failing Fulu tests
Update EF spec tests

beacon_node/beacon_chain/src/fetch_blobs.rs

consensus/types/src/lib.rs

mergify · 2025-03-13T05:47:36Z

This pull request has merge conflicts. Could you please resolve them @jimmygchen? 🙏

consensus/types/src/eth_spec.rs

…obsAndProof`.

# Conflicts: # beacon_node/beacon_chain/src/data_availability_checker.rs # beacon_node/beacon_chain/src/fetch_blobs.rs # beacon_node/http_api/src/publish_blocks.rs

mergify · 2025-04-04T09:02:25Z

This pull request has merge conflicts. Could you please resolve them @jimmygchen? 🙏

…ead of just custody columns.

beacon_node/beacon_chain/src/fetch_blobs.rs

# Conflicts: # beacon_node/beacon_chain/src/fetch_blobs.rs

…roofs` max size to be same as spec.

Squashed commit of the following: commit a5f9649 Author: Jimmy Chen <[email protected]> Date: Mon Apr 7 23:17:23 2025 +1000 Update error logging when proofs len is inconsistent and update `KzgProofs` max size to be same as spec. commit 68bb317 Author: Jimmy Chen <[email protected]> Date: Mon Apr 7 22:37:24 2025 +1000 Revive block root logging to fetch blobs function. commit e47032e Merge: d6197be 47a85cd Author: Jimmy Chen <[email protected]> Date: Mon Apr 7 22:35:07 2025 +1000 Merge remote-tracking branch 'origin/unstable' into cell_proofs # Conflicts: # beacon_node/beacon_chain/src/fetch_blobs.rs commit d6197be Author: Jimmy Chen <[email protected]> Date: Mon Apr 7 18:31:19 2025 +1000 Remove data column receiver and fix getBlobs storing all columns instead of just custody columns. commit e6f1c11 Author: Jimmy Chen <[email protected]> Date: Fri Apr 4 16:54:14 2025 +1100 Remove blob KZG proof computation in `getBlobSidecars` API. commit 1fbf643 Merge: 089338d 0e6da0f Author: Jimmy Chen <[email protected]> Date: Fri Apr 4 16:50:09 2025 +1100 Merge branch 'unstable' into cell_proofs commit 089338d Merge: b92f978 3f6c11d Author: Jimmy Chen <[email protected]> Date: Tue Mar 25 23:18:07 2025 +1100 Merge remote-tracking branch 'origin/unstable' into cell_proofs # Conflicts: # beacon_node/beacon_chain/src/data_availability_checker.rs # beacon_node/beacon_chain/src/fetch_blobs.rs # beacon_node/http_api/src/publish_blocks.rs commit b92f978 Author: Jimmy Chen <[email protected]> Date: Tue Mar 25 23:11:06 2025 +1100 Revert `BlobsBundle` field name change and rename proofs field in `BlobsAndProof`. commit 898e3fa Author: Jimmy Chen <[email protected]> Date: Thu Mar 13 00:37:59 2025 +1100 Remove blob proof validation under PeerDAS. commit 67e92b7 Author: Jimmy Chen <[email protected]> Date: Wed Mar 12 16:45:02 2025 +1100 Add support for getPayloadV5 and getBlobsV2. commit 0e9a400 Author: Mark Mackey <[email protected]> Date: Tue Mar 11 19:09:21 2025 -0500 just getting started..

beacon_node/beacon_chain/src/beacon_chain.rs

beacon_node/beacon_chain/src/data_column_verification.rs

beacon_node/beacon_chain/src/fetch_blobs.rs

beacon_node/beacon_chain/src/kzg_utils.rs

beacon_node/network/src/network_beacon_processor/mod.rs

consensus/types/src/eth_spec.rs

beacon_node/beacon_chain/src/fetch_blobs.rs

beacon_node/network/src/network_beacon_processor/mod.rs

consensus/types/src/eth_spec.rs

mergify · 2025-04-08T06:01:10Z

This pull request has merge conflicts. Could you please resolve them @jimmygchen? 🙏

dapplion

LGTM!

# Conflicts: # beacon_node/network/src/network_beacon_processor/mod.rs

ethDreamer and others added 2 commits March 11, 2025 19:09

just getting started..

0e9a400

Add support for getPayloadV5 and getBlobsV2.

67e92b7

jimmygchen added das Data Availability Sampling fulu Required for the upcoming Fulu hard fork work-in-progress PR is a work-in-progress labels Mar 12, 2025

jimmygchen assigned jimmygchen and ethDreamer Mar 12, 2025

Remove blob proof validation under PeerDAS.

898e3fa

dapplion reviewed Mar 12, 2025

View reviewed changes

beacon_node/beacon_chain/src/fetch_blobs.rs Outdated Show resolved Hide resolved

consensus/types/src/lib.rs Show resolved Hide resolved

jimmygchen commented Mar 17, 2025

View reviewed changes

consensus/types/src/eth_spec.rs Outdated Show resolved Hide resolved

jimmygchen mentioned this pull request Mar 25, 2025

Add Electra forks to basic sim tests #7199

Merged

jimmygchen added 2 commits March 25, 2025 23:11

Revert BlobsBundle field name change and rename proofs field in `Bl…

b92f978

…obsAndProof`.

Merge remote-tracking branch 'origin/unstable' into cell_proofs

089338d

# Conflicts: # beacon_node/beacon_chain/src/data_availability_checker.rs # beacon_node/beacon_chain/src/fetch_blobs.rs # beacon_node/http_api/src/publish_blocks.rs

jimmygchen mentioned this pull request Mar 26, 2025

Only allow one data column computation at a time for a given block #6764

Open

jimmygchen added 2 commits April 4, 2025 16:50

Merge branch 'unstable' into cell_proofs

1fbf643

Remove blob KZG proof computation in getBlobSidecars API.

e6f1c11

jimmygchen mentioned this pull request Apr 7, 2025

Fullnodes to publish data columns from EL getBlobs #7258

Merged

Remove data column receiver and fix getBlobs storing all columns inst…

d6197be

…ead of just custody columns.

jimmygchen requested a review from jxs as a code owner April 7, 2025 08:31

jimmygchen commented Apr 7, 2025

View reviewed changes

beacon_node/beacon_chain/src/fetch_blobs.rs Show resolved Hide resolved

jimmygchen added ready-for-review The code is ready for review and removed work-in-progress PR is a work-in-progress labels Apr 7, 2025

jimmygchen added 3 commits April 7, 2025 22:35

Merge remote-tracking branch 'origin/unstable' into cell_proofs

e47032e

# Conflicts: # beacon_node/beacon_chain/src/fetch_blobs.rs

Revive block root logging to fetch blobs function.

68bb317

Update error logging when proofs len is inconsistent and update `KzgP…

a5f9649

…roofs` max size to be same as spec.

jimmygchen force-pushed the cell_proofs branch from 7f91ef8 to a5f9649 Compare April 7, 2025 13:21

Fix tests and update checkpoint sync to compute cell proofs.

57e5389

jimmygchen added the peerdas-devnet-6 label Apr 7, 2025

dapplion reviewed Apr 7, 2025

View reviewed changes

jimmygchen force-pushed the cell_proofs branch from eef88dd to fc18183 Compare April 8, 2025 02:18

Add KZG verification during block production.

7f65c50

jimmygchen force-pushed the cell_proofs branch from fc18183 to 7f65c50 Compare April 8, 2025 02:24

jimmygchen added 2 commits April 8, 2025 12:29

Rename method based on review comments.

73fe317

Address more review comments.

5ce8299

dapplion reviewed Apr 8, 2025

View reviewed changes

Address review comments.

5a264f7

dapplion approved these changes Apr 8, 2025

View reviewed changes

jimmygchen added 2 commits April 8, 2025 16:04

Merge remote-tracking branch 'origin/unstable' into cell_proofs

e6595ce

# Conflicts: # beacon_node/network/src/network_beacon_processor/mod.rs

Fix build and merge conflicts.

9eef0e1

jimmygchen added ready-for-merge This PR is ready to merge. and removed ready-for-review The code is ready for review labels Apr 8, 2025

mergify bot merged commit 759b061 into sigp:unstable Apr 8, 2025
31 checks passed

jimmygchen mentioned this pull request Apr 8, 2025

Offloading KZG Proof Computation from the beacon node #7108

Closed

jimmygchen deleted the cell_proofs branch April 8, 2025 08:00

jimmygchen mentioned this pull request May 1, 2025

Filter out the downloaded/computed sample columns and only store the custody columns #6465

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Offloading KZG Proof Computation from the beacon node #7117

Offloading KZG Proof Computation from the beacon node #7117

Uh oh!

jimmygchen commented Mar 12, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

mergify bot commented Mar 13, 2025

Uh oh!

Uh oh!

mergify bot commented Apr 4, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mergify bot commented Apr 8, 2025

Uh oh!

dapplion left a comment

Uh oh!

Uh oh!

Uh oh!

Offloading KZG Proof Computation from the beacon node #7117

Offloading KZG Proof Computation from the beacon node #7117

Uh oh!

Conversation

jimmygchen commented Mar 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issue Addressed

Tasks

Uh oh!

Uh oh!

Uh oh!

mergify bot commented Mar 13, 2025

Uh oh!

Uh oh!

mergify bot commented Apr 4, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mergify bot commented Apr 8, 2025

Uh oh!

dapplion left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jimmygchen commented Mar 12, 2025 •

edited

Loading