ERC-7813 Table-based onchain storage (Store, Table-Based Introspectable Storage)
Final · Other · created 2024-11-08
A storage standard that saves smart contract data in tables like a database and emits a standard event whenever it changes, so that offchain services can automatically replicate the state.
Why it is needed
Smart contracts usually keep storage variables in a fixed internal layout, and to read them from outside you have to write a getter function for each one. As a result, every contract exposes its data in a different way, and services that collect and organize data off-chain (indexers) must write custom code for each individual contract. Because the storage structure is fixed in the code, changing how data is organized or adding a new field easily breaks existing integrations, and the fixed storage layout also makes upgrading contract logic risky. This standard reduces these problems by putting data into predefined structures (tables), like a database, and having contracts emit notifications (events) in a uniform format whenever the data changes. It also defines an efficient data compression scheme, which saves the fees (gas) spent on storage and event emission.
What to implement
event Store_DeleteRecord(ResourceId indexed tableId, bytes32[] keyTuple);: The event emitted when a record is deleted from a table.
In depth
Tables and ResourceId
Every table is identified by a 32-byte ResourceId. The first 2 bytes indicate the table type: 0x7462 ("tb") is an on-chain table and 0x6f74 ("ot") is an off-chain table. The remaining 30 bytes are a unique identifier. An on-chain table stores its state on the chain and also emits events. An off-chain table does not store its state on the chain and only emits events for indexers. A record is specified by a keyTuple made up of one or more keys.
Schema, FieldLayout, EncodedLengths
Each table has a key schema and a value schema. The key schema allows only fixed-length types, while the value schema can mix fixed-length and variable-length types. A Schema fits in a single 32 bytes: bytes 0-1 hold the total byte length of the fixed fields, byte 2 holds the number of fixed fields, byte 3 holds the number of variable-length fields, and bytes 4-31 hold the SchemaType of each field. Variable-length types must come after fixed-length types. FieldLayout is an optimization type that stores the precomputed byte length of each fixed field, which removes the cost of repeatedly computing lengths from the schema at runtime. EncodedLengths packs the lengths of a record's variable-length fields into a single 32 bytes. The total length is encoded as a uint56, and each of the five per-field lengths is encoded as a uint40.
Field limits and packed encoding
A record can have at most 28 fields in total, fixed and variable-length combined, and at most 5 of them can be variable-length. The maximum number of fixed fields is 28 minus the number of variable-length fields. The encoding is tightly packed with no padding. Fixed fields are concatenated in schema order like abi.encodePacked, and array elements are also concatenated without padding. Variable-length fields are concatenated without length information, and their lengths are collected in a single EncodedLengths word. The whole record is represented as a (staticData, encodedLengths, dynamicData) tuple, and this scheme reduces the gas cost of storage and event emission.
The IStore read interface
A Store must implement three read functions. getRecord returns the whole record as an encoded tuple given a table ID and a key tuple, getField returns the single field specified by fieldIndex, encoded on its own, and getFieldLength returns the byte length of that field. The standard does not define write functions, and each implementation designs them to fit its purpose.
Four events and the consistency requirement
A defined event must be emitted for every state change. Store_SetRecord is emitted when a whole record is overwritten. Store_SpliceStaticData is emitted when bytes are replaced starting at the start position of the static data (the total length of the static data does not change, because the value schema determines it). Store_SpliceDynamicData is emitted when bytes are deleted, replaced or inserted in the dynamic data (it includes dynamicFieldIndex, start, deleteCount and the resulting encodedLengths). Store_DeleteRecord is emitted when a record is removed. For an on-chain table, the data returned by the read functions at any block must match the result of applying the events up to that block in order. This requirement lets an indexer reconstruct the state accurately from events alone.
Why Splice events exist and why arrays of variable-length types are banned
Tracking is possible with Store_SetRecord alone, but when only part of a record changes, reading the whole data from storage and emitting it again is inefficient. Splice events emit only the minimal data that changed, which saves gas, and even when a variable-length field is large, the cost of modifying it does not grow with the field size. On the other hand, arrays of variable-length types such as string[] or bytes[] are deliberately not supported. This is to enforce a flat schema and keep implementations simple and efficient. If needed, you can create a separate table with a schema such as { index: uint256, data: bytes } and store each array element as one record.
The Tables table that also describes itself
A Store must include the Tables table, a special on-chain table that holds the metadata of all tables. Its key is a single tableId, and its value has five fields: fieldLayout, keySchema, valueSchema, abiEncodedKeyNames and abiEncodedFieldNames. A Store_SetRecord event must be emitted once each time a table is registered, and the registered record is treated as immutable, so no other Store events may be emitted for it afterwards. Also, before any other table is registered, the record describing the Tables table itself must be stored first under the predefined tableId (0x746273746f72650000000000000000005461626c657300000000000000000000). Because the ResourceId and schema are predefined, an indexer can interpret the schema of every table and build schema-aware APIs such as SQL-like queries.
Things to watch out for
- Write access control is the implementation's responsibility: This standard defines only the read functions (getRecord, getField, getFieldLength) and leaves the way records are written or modified to the implementation. Each implementation must therefore provide write access control suited to its purpose.
- Stored data is fully public on-chain: Data stored in a Store can be read directly through the read functions not only off-chain but also by other smart contracts. Developers should keep in mind that all data placed in a Store is open to other contracts. This differs from ordinary contracts, where internal storage variables are private to other contracts by default.