EIP-7495 SSZ progressive container (SSZ ProgressiveContainer)

No call decision · Consensus & staking · EIP status Review · created 2023-08-18

Adds a new SSZ container type whose per-field positions in the Merkle tree remain fixed even as fields are added or removed. Existing Merkle proof verifiers continue to work even when the data structure changes across forks.

Why it is needed

Data structures in Ethereum's consensus layer are serialized using a format called SSZ (Simple Serialize) and summarized as a Merkle tree. However, as fields are added or removed with each fork, the data structure's version changes, and if the field count exceeds a power of two or a field is removed or replaced, the shape of the Merkle tree itself changes. This breaks every program that was verifying Merkle proofs based on that tree. Fixing a smart contract verifier may require a security committee to upgrade its logic, and embedded devices may need a firmware update. This work is needed even when there is no meaningful change to the fields a given verifier actually cares about. This EIP solves the problem by creating a new container type whose field positions stay fixed even when fields change.

What changes

Today

After EIP-7495

Who is affected

In depth

ProgressiveContainer(active_fields) type definition

A new SSZ composite type that holds an ordered collection of heterogeneous values while guaranteeing stable merkleization. active_fields is a sequence of 1s and 0s, and actual fields are placed in order only at the positions marked 1. In the original's example, Square uses active_fields=[1, 0, 1], placing side(uint16) at field index 0 and color(uint8) at index 2, while Circle uses active_fields=[0, 1, 1], placing radius(uint16) at index 1 and color(uint8) at index 2. Although they are different types, color keeps the same position. The default value is the list of default values of each field's type.

Illegal type rules

Four configurations are banned as illegal types: a ProgressiveContainer with no fields at all, active_fields with more than 256 entries, active_fields ending in 0, and active_fields whose count of 1s differs from the actual number of fields. Empty containers are banned because their serialized length would be 0, making it impossible to tell from serialization alone how many elements a list of such containers holds.

Serialization, deserialization, and JSON mapping

Serialization and deserialization are exactly the same as for an existing Container. Only the active fields are packed in order, so missing fields occupy no bytes at all. JSON mapping is also defined as a regular object, just like existing containers. The only part that differs from existing types is merkleization.

Merkleization: the progressive tree and the active_fields mixin

Two helper functions are added to the merkleization spec. get_active_fields(value) returns the active_fields of that type, and mix_in_active_fields takes a merkle root root and active_fields and computes hash(root, pack_bits(active_fields)). active_fields is capped at 256 bits. The full root is defined as mix_in_active_fields(merkleize_progressive(field_roots), get_active_fields(value)), where field_roots holds the corresponding field's hash_tree_root at each active position and Bytes32() at each inactive position. The tree structure uses the progressive merkle tree from EIP-7916, so it grows with minimal overhead matched to the actual number of fields.

Why cap it at 256 bits

Capping active_fields at 256 bits (one word) keeps the mixin simple, keeps it consistent with the length mixin used for lists, and is practically sufficient. An alternative of using a ProgressiveBitlist as the mixin was also considered, but it would effectively require caching the mixin's precomputed hash_tree_root and would complicate verifier logic, so it was judged over-engineered. The 256 limit also counts deprecated fields, but the original notes that since the current BeaconState has only around 40 fields, approaching that limit is unlikely. If that situation does arise, a 'more' field holding a nested ProgressiveContainer can be added.

Security consideration: distinguishing an absent field from a zero value

Light-client-based verifiers and smart contracts (for example, those built on EIP-4788) are not updated on the same cadence as Ethereum itself. Even if a future fork removes some field, the active_fields mixin lets such verifiers distinguish between a field being absent and a field having a value of zero. Without active_fields, the hash_tree_root of the two cases would collide. The new type also does not collide with existing SSZ types, so there is no backward compatibility issue.

Status and remaining work

This EIP's status is Review, so the spec may still change. Introducing Optional[type] was deferred to a future EIP since it is not needed for current functionality, and both marking optional fields via the active_fields bitvector and sparse list serialization remain open areas for future exploration. Tests live in ethereum/remerkleable's test_impl.py and test_typing.py and in consensus-specs' ssz_generic tests, and the reference implementation is ethereum/remerkleable.

Original EIP · ← EIPs