Skip to content

Commit 16b8113

Browse files
Electra epoch processing (#5761)
* Attestation superstruct changes for EIP 7549 (#5644) * update * experiment * superstruct changes * revert * superstruct changes * fix tests * indexed attestation * indexed attestation superstruct * updated TODOs * `superstruct` the `AttesterSlashing` (#5636) * `superstruct` Attester Fork Variants * Push a little further * Deal with Encode / Decode of AttesterSlashing * not so sure about this.. * Stop Encode/Decode Bounds from Propagating Out * Tons of Changes.. * More Conversions to AttestationRef * Add AsReference trait (#15) * Add AsReference trait * Fix some snafus * Got it Compiling! :D * Got Tests Building * Get beacon chain tests compiling --------- Co-authored-by: Michael Sproul <[email protected]> * Merge remote-tracking branch 'upstream/unstable' into electra_attestation_changes * Make EF Tests Fork-Agnostic (#5713) * Finish EF Test Fork Agnostic (#5714) * Superstruct `AggregateAndProof` (#5715) * Upgrade `superstruct` to `0.8.0` * superstruct `AggregateAndProof` * Merge remote-tracking branch 'sigp/unstable' into electra_attestation_changes * cargo fmt * Merge pull request #5726 from realbigsean/electra_attestation_changes Merge unstable into Electra attestation changes * process withdrawals updates * cleanup withdrawals processing * update `process_operations` deposit length check * add apply_deposit changes * add execution layer withdrawal request processing * process deposit receipts * add consolidation processing * update process operations function * exit updates * clean up * update slash_validator * EIP7549 `get_attestation_indices` (#5657) * get attesting indices electra impl * fmt * get tests to pass * fmt * fix some beacon chain tests * fmt * fix slasher test * fmt got me again * fix more tests * fix tests * Some small changes (#5739) * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * cargo fmt (#5740) * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * fix attestation verification * Sketch op pool changes * fix get attesting indices (#5742) * fix get attesting indices * better errors * fix compile * only get committee index once * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Ef test fixes (#5753) * attestation related ef test fixes * delete commented out stuff * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Fix Aggregation Pool for Electra (#5754) * Fix Aggregation Pool for Electra * Remove Outdated Interface * fix ssz (#5755) * Get `electra_op_pool` up to date (#5756) * fix get attesting indices (#5742) * fix get attesting indices * better errors * fix compile * only get committee index once * Ef test fixes (#5753) * attestation related ef test fixes * delete commented out stuff * Fix Aggregation Pool for Electra (#5754) * Fix Aggregation Pool for Electra * Remove Outdated Interface * fix ssz (#5755) --------- Co-authored-by: realbigsean <[email protected]> * Revert "Get `electra_op_pool` up to date (#5756)" (#5757) This reverts commit ab9e58a. * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into electra_op_pool * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Compute on chain aggregate impl (#5752) * add compute_on_chain_agg impl to op pool changes * fmt * get op pool tests to pass * update the naive agg pool interface (#5760) * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Fix bugs in cross-committee aggregation * Add comment to max cover optimisation * Fix assert * Electra epoch processing * Merge pull request #5749 from sigp/electra_op_pool Optimise Electra op pool aggregation * don't fail on empty consolidations * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * update committee offset * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * update committee offset * only increment the state deposit index on old deposit flow * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * use correct max eb in epoch cache initialization * drop initiate validator ordering optimization * fix initiate exit for single pass * Fix Electra Fork Choice Tests (#5764) * Fix Electra Fork Choice Tests (#5764) * Fix Electra Fork Choice Tests (#5764) * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Fix Consolidation Sigs & Withdrawals * Merge pull request #5766 from ethDreamer/two_fixes Fix Consolidation Sigs & Withdrawals * Merge branches 'block-processing-electra' and 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-epoch-proc * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * fix slashing handling * Fix Bug In Block Processing with 0x02 Credentials * Merge remote-tracking branch 'upstream/unstable' * Send unagg attestation based on fork * Publish all aggregates * just one more check bro plz.. * Merge pull request #5832 from ethDreamer/electra_attestation_changes_merge_unstable Merge `unstable` into `electra_attestation_changes` * Merge pull request #5835 from realbigsean/fix-validator-logic Fix validator logic * Merge pull request #5816 from realbigsean/electra-attestation-slashing-handling Electra slashing handling * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Electra attestation changes rm decode impl (#5856) * Remove Crappy Decode impl for Attestation * Remove Inefficient Attestation Decode impl * Implement Schema Upgrade / Downgrade * Update beacon_node/beacon_chain/src/schema_change/migration_schema_v20.rs Co-authored-by: Michael Sproul <[email protected]> --------- Co-authored-by: Michael Sproul <[email protected]> * Fix failing attestation tests and misc electra attestation cleanup (#5810) * - get attestation related beacon chain tests to pass - observed attestations are now keyed off of data + committee index - rename op pool attestationref to compactattestationref - remove unwraps in agg pool and use options instead - cherry pick some changes from ef-tests-electra * cargo fmt * fix failing test * Revert dockerfile changes * make committee_index return option * function args shouldnt be a ref to attestation ref * fmt * fix dup imports --------- Co-authored-by: realbigsean <[email protected]> * fix some todos (#5817) * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * add consolidations to merkle calc for inclusion proof * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Remove Duplicate KZG Commitment Merkle Proof Code (#5874) * Remove Duplicate KZG Commitment Merkle Proof Code * s/tree_lists/fields/ * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * fix compile * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Fix slasher tests (#5906) * Fix electra tests * Add electra attestations to double vote tests * Update superstruct to 0.8 * Merge remote-tracking branch 'origin/unstable' into electra_attestation_changes * Small cleanup in slasher tests * Clean up Electra observed aggregates (#5929) * Use consistent key in observed_attestations * Remove unwraps from observed aggregates * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * De-dup attestation constructor logic * Remove unwraps in Attestation construction * Dedup match_attestation_data * Remove outdated TODO * Use ForkName Ord in fork-choice tests * Use ForkName Ord in BeaconBlockBody * Make to_electra not fallible * Remove TestRandom impl for IndexedAttestation * Remove IndexedAttestation faulty Decode impl * Drop TestRandom impl * Add PendingAttestationInElectra * Indexed att on disk (#35) * indexed att on disk * fix lints * Update slasher/src/migrate.rs Co-authored-by: ethDreamer <[email protected]> --------- Co-authored-by: Lion - dapplion <[email protected]> Co-authored-by: ethDreamer <[email protected]> * add electra fork enabled fn to ForkName impl (#36) * add electra fork enabled fn to ForkName impl * remove inadvertent file * Update common/eth2/src/types.rs Co-authored-by: ethDreamer <[email protected]> * Dedup attestation constructor logic in attester cache * Use if let Ok for committee_bits * Dedup Attestation constructor code * Diff reduction in tests * Fix beacon_chain tests * Diff reduction * Use Ord for ForkName in pubsub * Resolve into_attestation_and_indices todo * Remove stale TODO * Fix beacon_chain tests * Test spec invariant * Use electra_enabled in pubsub * Remove get_indexed_attestation_from_signed_aggregate * Use ok_or instead of if let else * committees are sorted * remove dup method `get_indexed_attestation_from_committees` * Merge pull request #5940 from dapplion/electra_attestation_changes_lionreview Electra attestations #5712 review * update default persisted op pool deserialization * ensure aggregate and proof uses serde untagged on ref * Fork aware ssz static attestation tests * Electra attestation changes from Lions review (#5971) * dedup/cleanup and remove unneeded hashset use * remove irrelevant TODOs * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Electra attestation changes sean review (#5972) * instantiate empty bitlist in unreachable code * clean up error conversion * fork enabled bool cleanup * remove a couple todos * return bools instead of options in `aggregate` and use the result * delete commented out code * use map macros in simple transformations * remove signers_disjoint_from * get ef tests compiling * get ef tests compiling * update intentionally excluded files * Avoid changing slasher schema for Electra * Delete slasher schema v4 * Fix clippy * Fix compilation of beacon_chain tests * Update database.rs * Update per_block_processing.rs * Add electra lightclient types * Update slasher/src/database.rs * fix imports * Merge pull request #5980 from dapplion/electra-lightclient Add electra lightclient types * Merge pull request #5975 from michaelsproul/electra-slasher-no-migration Avoid changing slasher schema for Electra * Update beacon_node/beacon_chain/src/attestation_verification.rs * Update beacon_node/beacon_chain/src/attestation_verification.rs * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * Merge branch 'electra_attestation_changes' of https://github.com/realbigsean/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-epoch-proc * Update consensus/state_processing/src/per_epoch_processing/single_pass.rs * Update consensus/state_processing/src/per_epoch_processing/single_pass.rs * Update consensus/state_processing/src/per_epoch_processing/single_pass.rs * Update consensus/state_processing/src/per_epoch_processing/single_pass.rs * Update consensus/state_processing/src/per_epoch_processing/single_pass.rs
1 parent a64cee3 commit 16b8113

File tree

6 files changed

+507
-71
lines changed

6 files changed

+507
-71
lines changed

consensus/state_processing/src/common/initiate_validator_exit.rs

Lines changed: 7 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -8,12 +8,12 @@ pub fn initiate_validator_exit<E: EthSpec>(
88
index: usize,
99
spec: &ChainSpec,
1010
) -> Result<(), Error> {
11-
// We do things in a slightly different order to the spec here. Instead of immediately checking
12-
// whether the validator has already exited, we instead prepare the exit cache and compute the
13-
// cheap-to-calculate values from that. *Then* we look up the validator a single time in the
14-
// validator tree (expensive), make the check and mutate as appropriate. Compared to the spec
15-
// ordering, this saves us from looking up the validator in the validator registry multiple
16-
// times.
11+
let validator = state.get_validator_cow(index)?;
12+
13+
// Return if the validator already initiated exit
14+
if validator.exit_epoch != spec.far_future_epoch {
15+
return Ok(());
16+
}
1717

1818
// Ensure the exit cache is built.
1919
state.build_exit_cache(spec)?;
@@ -36,14 +36,7 @@ pub fn initiate_validator_exit<E: EthSpec>(
3636
exit_queue_epoch
3737
};
3838

39-
let validator = state.get_validator_cow(index)?;
40-
41-
// Return if the validator already initiated exit
42-
if validator.exit_epoch != spec.far_future_epoch {
43-
return Ok(());
44-
}
45-
46-
let validator = validator.into_mut()?;
39+
let validator = state.get_validator_mut(index)?;
4740
validator.exit_epoch = exit_queue_epoch;
4841
validator.withdrawable_epoch =
4942
exit_queue_epoch.safe_add(spec.min_validator_withdrawability_delay)?;

consensus/state_processing/src/epoch_cache.rs

Lines changed: 41 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ use types::{ActivationQueue, BeaconState, ChainSpec, EthSpec, ForkName, Hash256}
99
pub struct PreEpochCache {
1010
epoch_key: EpochCacheKey,
1111
effective_balances: Vec<u64>,
12+
total_active_balance: u64,
1213
}
1314

1415
impl PreEpochCache {
@@ -36,27 +37,59 @@ impl PreEpochCache {
3637
Ok(Self {
3738
epoch_key,
3839
effective_balances: Vec::with_capacity(state.validators().len()),
40+
total_active_balance: 0,
3941
})
4042
}
4143

42-
pub fn push_effective_balance(&mut self, effective_balance: u64) {
43-
self.effective_balances.push(effective_balance);
44+
pub fn update_effective_balance(
45+
&mut self,
46+
validator_index: usize,
47+
effective_balance: u64,
48+
is_active_next_epoch: bool,
49+
) -> Result<(), EpochCacheError> {
50+
if validator_index == self.effective_balances.len() {
51+
self.effective_balances.push(effective_balance);
52+
if is_active_next_epoch {
53+
self.total_active_balance
54+
.safe_add_assign(effective_balance)?;
55+
}
56+
57+
Ok(())
58+
} else if let Some(existing_balance) = self.effective_balances.get_mut(validator_index) {
59+
// Update total active balance for a late change in effective balance. This happens when
60+
// processing consolidations.
61+
if is_active_next_epoch {
62+
self.total_active_balance
63+
.safe_add_assign(effective_balance)?;
64+
self.total_active_balance
65+
.safe_sub_assign(*existing_balance)?;
66+
}
67+
*existing_balance = effective_balance;
68+
Ok(())
69+
} else {
70+
Err(EpochCacheError::ValidatorIndexOutOfBounds { validator_index })
71+
}
72+
}
73+
74+
pub fn get_total_active_balance(&self) -> u64 {
75+
self.total_active_balance
4476
}
4577

4678
pub fn into_epoch_cache(
4779
self,
48-
total_active_balance: u64,
4980
activation_queue: ActivationQueue,
5081
spec: &ChainSpec,
5182
) -> Result<EpochCache, EpochCacheError> {
5283
let epoch = self.epoch_key.epoch;
84+
let total_active_balance = self.total_active_balance;
5385
let sqrt_total_active_balance = SqrtTotalActiveBalance::new(total_active_balance);
5486
let base_reward_per_increment = BaseRewardPerIncrement::new(total_active_balance, spec)?;
5587

5688
let effective_balance_increment = spec.effective_balance_increment;
57-
let max_effective_balance_eth = spec
58-
.max_effective_balance
59-
.safe_div(effective_balance_increment)?;
89+
let max_effective_balance =
90+
spec.max_effective_balance_for_fork(spec.fork_name_at_epoch(epoch));
91+
let max_effective_balance_eth =
92+
max_effective_balance.safe_div(effective_balance_increment)?;
6093

6194
let mut base_rewards = Vec::with_capacity(max_effective_balance_eth.safe_add(1)? as usize);
6295

@@ -131,9 +164,9 @@ pub fn initialize_epoch_cache<E: EthSpec>(
131164
decision_block_root,
132165
},
133166
effective_balances,
167+
total_active_balance,
134168
};
135-
*state.epoch_cache_mut() =
136-
pre_epoch_cache.into_epoch_cache(total_active_balance, activation_queue, spec)?;
169+
*state.epoch_cache_mut() = pre_epoch_cache.into_epoch_cache(activation_queue, spec)?;
137170

138171
Ok(())
139172
}

consensus/state_processing/src/per_epoch_processing/errors.rs

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,9 @@ pub enum EpochProcessingError {
2525
InvalidFlagIndex(usize),
2626
MilhouseError(milhouse::Error),
2727
EpochCache(EpochCacheError),
28+
SinglePassMissingActivationQueue,
29+
MissingEarliestExitEpoch,
30+
MissingExitBalanceToConsume,
2831
}
2932

3033
impl From<InclusionError> for EpochProcessingError {

0 commit comments

Comments
 (0)