EIP-7801 etha sharded block subprotocol (etha - Sharded Blocks Subprotocol)
No call decision · Networking & nodes · EIP status Draft · created 2024-10-30
Introduces a new communication protocol, etha, in which a node splits its historical block record into segments, stores only some of them, and advertises which segments it holds via a bitmask immediately upon connecting.
Why it is needed
Once EIP-4444 is applied, nodes can delete old block data, and only some nodes will keep serving historical data going forward. But there is no way to know in advance which node holds which data. Under the current structure, a node must connect and request a block just to find out whether the peer has the data, and this wastes bandwidth unnecessarily. This proposal divides chain history into segments of 106,496 blocks and has nodes advertise which segments they hold, at connection time, as a set of on/off flags called a bitmask. This lets peers determine who has what data before sending a request. It leaves the existing eth protocol untouched by splitting this out into a separate subprotocol, which also prevents requests from flooding nodes that do not hold the relevant historical data.
What changes
Today
- To find out whether a peer holds historical block data, you currently have to connect and request it first.
- Requests go to nodes that don't have the data, wasting bandwidth.
- There is no standard way for a node that has pruned its history to advertise the range it still holds.
After EIP-7801
- The holder's segments are advertised as a bitmask right at the handshake immediately after connecting.
- Peers choose which node to request from before sending the request, based on who holds the data.
- A node may store as little as the minimum, 10% of the full history, in segments.
- A node commits to continuing to retain the segments it advertised even as new blocks accumulate.
Who is affected
- Node operators: Storage space can be cut by about 90% compared to storing the full history, and future storage growth also increases at only 10% of the rate it otherwise would.
- Client teams: A separate etha subprotocol, independent of the eth protocol, must be implemented. The source material holds that the full history is only sufficiently replicated across the network if multiple clients adopt it as a default.
- Apps and services that need historical data: Even as nodes prune their records, each segment remains distributed across the network as a whole, so historical blocks and receipts can still be retrieved.
- Validators / consensus layer: No effect on the consensus engine, and no hard fork is required.
In depth
Handshake packet and bitmask structure
The handshake packet is [version: P, networkid: P, blockhash: B_32, genesis: B_32, forkid, blockBitmask]. blockBitmask is a 10-bit bitmask, where within a range of 1,064,960 blocks each bit represents one 106,496-block segment. This bitmask repeats every 1,064,960 blocks, which allows data availability probability across the whole chain to be reasoned about simply. All elements other than blockBitmask are the same as defined in eth/69. Since the held segments are conveyed in a single handshake, no additional message type is needed.
Four supported message types carried over from eth/69
etha must support four messages from eth/69 to serve historical data: GetBlockBodies (0x05), BlockBodies (0x06), GetReceipts (0x0f), and Receipts (0x10). Their meaning and payload structure are identical to the corresponding messages in eth/69, so existing implementations can be reused as is to maintain compatibility.
Node behavior rules
On startup, a node MAY turn on at least one bit of its blockBitmask and backfill the corresponding 106,496-block segment. It MUST retain new block segments according to its bitmask, aiming to cover at least 10% of chain history. It also MUST keep retaining the segments corresponding to its advertised bitmask as new blocks are added. Requiring commitment to future segments as well is meant to increase the predictability and stability of data placement.
Proposed alternative: an ENR extension
Instead of a handshake, blockBitmask could also be encoded in, or derived from, an Ethereum Node Record (ENR). For example, the bitmask could be derived from the ENR's secp256k1 field. This would allow advertising held segments without connecting, but the source material notes it lacks the authentication and reliability that the handshake approach provides, and gives no guarantee that a connecting node even supports etha.
Segment size and design rationale
The range sizes of 106,496 and 1,064,960 were chosen because they are multiples of 8,192, the maximum block range of an Era1 file. This simplifies storage and representation via Era1 files. The approach of advertising held ranges with a bitlist is already used in the consensus layer's attestation subnets, making it a familiar and efficient method, the source material explains. Separating etha from eth is also meant to prevent requests for historical ranges a node does not hold from flooding the eth protocol.
Availability probability and storage burden
The probability that no one holds a particular shard is modeled as P = (0.9)^n, where n is the number of peers. Assuming nodes are randomly distributed, this is 7% at 25 peers and 3.4% at 32 peers. This calculation rests on the assumption that a substantial share of network nodes each serve at least one shard, and the source material holds that sufficient replication of the full history across the network would require default adoption by multiple clients. As history grows, the size of a retained shard grows too, but the roughly 90% storage saving relative to full retention is preserved.
Current status
This EIP is in Draft status, so the figures and structure in the specification may still change. Because it is independent of the eth protocol, it has no effect on the consensus engine and requires no hard fork. No decision on fork inclusion has come out of core developer calls yet.