EIP-7688 Forward-compatible consensus data structures (Forward Compatible Consensus Data Structures)
No call decision · Consensus & staking · EIP status Review · created 2024-04-15
Changes the beacon chain's consensus data structures to ProgressiveContainer and ProgressiveList, which preserve Merkle proof paths across forks. Serialization stays the same; only the Merkleization method changes.
Why it is needed
Ethereum consensus data is serialized under a scheme called SSZ (Simple Serialize) and then compressed into a single summary hash through a process called Merkleization. External programs verify proofs against this summary value to confirm that specific data has not been tampered with. For example, a decentralized staking pool's smart contract uses this to check whether a participating validator has been slashed (had its stake forfeited as a penalty). However, whenever a fork (network upgrade) adds or removes a field from the data structure, the Merkleization result changes, breaking existing verification code. Software whose update cycle differs from Ethereum's, such as smart contracts, hardware wallets, and mobile device operating systems, must keep patching their verification logic to match Ethereum's fork schedule. This EIP changes the consensus data structures so that, as long as the feature being verified itself doesn't change, existing proof verifiers keep working across forks. It also separates a list's maximum capacity from its schema, so later EIPs can adjust the limit alone without touching the type or the Merkle proofs.
What changes
Today
- When a field is added or removed in a fork, the Merkle proof path (gindex) changes
- Proof verification logic in smart contracts or hardware wallets must be updated with every fork
- Because a list's maximum capacity is fixed in the SSZ schema, changing the limit changes both the type and the proofs
After EIP-7688
- Each field and list element is assigned a stable gindex, so proofs remain valid despite unrelated changes
- As long as the feature being verified stays the same, verification code does not need to be fixed with every fork
- List limits become runtime checks, so later EIPs can adjust them without changing the type
Who is affected
- Smart contract developers: This reduces the maintenance burden of keeping contracts that verify consensus data proofs, such as staking pools, in sync with the fork schedule.
- Hardware wallets / mobile OS: Off-chain verifiers keep working even though their update cycle differs from Ethereum's.
- Client teams: The Merkleization implementation for major consensus types must be converted to ProgressiveContainer and ProgressiveList.
- Existing proof verifiers: Because the Merkle tree shape changes once, existing verifiers, including contracts on other blockchains and hardware wallets, need a one-time update.
In depth
Rules for converting a Container to a ProgressiveContainer
Container types expected to change across forks SHALL be redefined as ProgressiveContainer(active_fields=[1] * len(type.fields())). During conversion, each field is assigned a stable generalized index (gindex) that remains valid in later forks. When a fork adds a field, a 1 MUST be appended to the end of active_fields; when a fork removes a field, the corresponding bit MUST be set to 0. It is RECOMMENDED that compatibility rules be enforced in the unit test framework by defining types such as CompatibleUnion[fork_1.Foo, fork_2.Foo, ...]. Once a field name has been used, it cannot be reused to represent a different type later, and new fields can only be appended at the end. The original text explains that this turns what has so far been convention, such as Altair's replacement of the participation fields, Capella's addition of historical_summaries, and Electra's addition of deposit_requests_start_index, into a strict rule.
Converting List and Bitlist to progressive types
Existing List types have either had capacities far larger than are actually reached, or had their capacities change across forks. Lists whose capacity is meant to be dynamic or effectively unlimited SHALL be redefined as ProgressiveList[type], and Bitlists as ProgressiveBitlist. Capacity limits SHALL be removed from the schema, and application logic SHALL instead check the appropriate limit at runtime. This separates operational limits from the SSZ schema, letting later EIPs adjust limits without affecting the data type or Merkle proofs, and also enables cross-operation constraints such as a shared total signature-check budget. During conversion, each list element is also assigned a stable gindex, and existing elements' indices remain valid even when new elements are added.
Redefining network message size limits
For each affected libp2p gossip topic and req/resp chunk type, it is RECOMMENDED to define a maximum uncompressed serialized message size constant derived from the pre-conversion type's capacity. For variable-size messages where no such constant is defined, MAX_PAYLOAD_SIZE applies. Serialization itself and libp2p network message size limits are not changed by this EIP, and the message-rejection behavior previously derived from per-type SSZ size bounds is preserved through the newly defined per-message bound constants.
List of types being converted
The following types are converted to ProgressiveContainer: Attestation (aggregation_bits becomes ProgressiveBitlist), IndexedAttestation (attesting_indices becomes ProgressiveList), ExecutionPayloadHeader, ExecutionPayload (transactions and withdrawals become ProgressiveList; the MAX_TRANSACTIONS_PER_PAYLOAD limit of 1M is no longer enforced), ExecutionRequests (deposits, withdrawals, and consolidations become ProgressiveList), BeaconBlockBody (proposer_slashings, attester_slashings, attestations, deposits, voluntary_exits, and bls_to_execution_changes become ProgressiveList), BeaconState (validators, balances, previous_epoch_participation, current_epoch_participation, inactivity_scores, pending_deposits, pending_partial_withdrawals, and pending_consolidations become ProgressiveList). Transaction is redefined as ProgressiveByteList, and the MAX_BYTES_PER_TRANSACTION (1 GB) limit is no longer enforced. The blob_kzg_commitments, kzg_proofs, and column fields are also changed to ProgressiveList.
Types not converted, and why
Base types used inside ProgressiveContainer definitions, such as Slot, Epoch, Gwei, Root, BLSPubkey, Checkpoint, and AttestationData, are treated as having immutable merkleization. If a future fork needs to change these in an incompatible way, it SHALL define a new type and assign a new field name. Validator is deliberately kept as a Container. It has not changed since genesis and is not expected to change before the quantum-resistance transition, and since a quantum-resistance transition that introduces a new hash function would invalidate existing Merkle proofs anyway, the type can be freely restructured at that point; converting it now would also balloon per-validator hashing overhead by millions of additional hashes. historical_roots and historical_summaries are likewise kept as List. The former has been frozen since Capella, and the latter is actively used by verifiers that prove against hash_tree_root, so converting it now would add confusion or break verifiers without sufficient benefit. A future EIP could address merging the two into a single accumulator.
Retroactive Application and Backwards Compatibility
ProgressiveContainer and ProgressiveList are serialized the same way as Container and List, but the merkleization and hash_tree_root of the affected data structures change. Verifiers that process historical data predating this EIP must therefore continue to support the original merkleization scheme as well. Existing Merkle proof verifiers must be updated to support the new Merkle tree shape, including the corresponding verifiers in smart contracts on other blockchains and in hardware wallets.
Current status
This EIP is in Review status, so its contents may still change. It requires EIP-7495, EIP-7607, and EIP-7916 as prerequisites. The provided call records contain no decision on whether this EIP will be included in a fork.