EIP-627 Whisper specification
Final · Networking & nodes · EIP status Final · created 2017-05-05
Standardizes the message format of Whisper, Ethereum's P2P messaging protocol. It covers packet codes, envelope structure, and the proof-of-work and bloom filter exchange method to ensure compatibility between clients.
Why it is needed
Whisper is a ÐΞVp2p-based communication protocol through which Ethereum nodes exchange encrypted messages separately from the blockchain. At the time, the message format was described only in a wiki document, and clients built by different teams, such as go-ethereum and Parity, each implemented it independently. If the format is not fixed as a standard, different clients may interpret messages differently and fail to communicate with one another, and compatibility with existing nodes could break as the protocol evolves to new versions. This EIP replaces the existing wiki specification by fixing the packet codes and envelope format in a document, and establishes the rule that unknown codes are ignored without error, to ensure forward compatibility between different Whisper clients.
What changes
Today
- The Whisper message format exists only as a wiki document, with no official standard.
- Message interpretation can differ between clients, making compatibility hard to guarantee.
- There is no standard way to exchange proof-of-work requirements or topics of interest with peers.
After EIP-627
- Packet codes 0 through 127 are reserved for Whisper, and 4 mandatory codes are defined.
- The rule that unknown codes are ignored without error preserves compatibility with future versions.
- The PoW Requirement and Bloom Filter packets dynamically adjust the conditions for which messages are accepted.
Who is affected
- Client teams: Whisper implementations such as go-ethereum and Parity now follow the same packet codes and envelope format, ensuring interoperability between different implementations.
- Dapp developers: When building messaging features with Whisper, they can rely on a standard format that is compatible with any client.
- Mail server operators: Can implement Whisper mail servers and clients that request and deliver expired past messages using the reserved P2P Request (126) and P2P Message (127) codes.
In depth
Packet codes and forward compatibility
Whisper messages are transmitted as ÐΞVp2p wire protocol packets, encoded as an RLP array containing an integer packet code and a single object whose type depends on that code. Codes 0 through 127 are reserved for Whisper. The four mandatory codes are Status(0), Messages(1), PoW Requirement(2), and Bloom Filter(3), while P2P Request(126) and P2P Message(127) are optional but have reserved uses. A node that receives a packet with an unsupported code must ignore it rather than raising an error. This rule preserves forward compatibility with future versions.
Status and Messages packets
Status(0x00) is a list containing, in this order, an integer version, a floating-point PoW requirement, and a bloom filter. The bloom filter may be omitted, and if it is absent or nil, that node is treated as a full node receiving all messages. Status must be sent right after the initial handshake, before any other message. Messages(0x01) carries a (possibly empty) list of Whisper envelopes, and is used to send standard Whisper envelopes.
PoW Requirement and its calculation
PoW Requirement(0x02) carries a single 64-bit IEEE 754 floating-point value. qNAN, sNAN, INF, -INF, and negative numbers are not allowed. A node that receives this message must thereafter not forward messages with PoW lower than the specified value to the sender. PoW is the average number of iterations needed to find the current BestBit (the number of consecutive leading zero bits in the hash), divided by the message size and TTL: PoW = (2**BestBit) / (size * TTL). It is computed by taking the RLP of the envelope excluding env_nonce (short_rlp), appending env_nonce, taking the sha3 hash (pow_hash), and computing 2**leading_zeros(pow_hash) / (size * ttl). size is the size of the fully RLP-encoded envelope. The Rationale explains that it is better to let the network adjust the requirement itself rather than hardcoding a minimum PoW value.
Structure of the bloom filter
Bloom Filter(0x03) is an arbitrary-length byte array used to tell peers which topics' messages a node is interested in. Because it conveys a range rather than revealing the exact topics of interest, it does not significantly compromise privacy, and additional bits can be used to tune how much information is conveyed and how efficient the filter is. The bloom is built by OR-ing together the per-topic bloom results. The bloom function projects a 4-byte topic onto a 512-bit slice, marking at most 3 bits per topic. The projection works by taking each byte value n from S[0], S[1], S[2], and, if the i-th bit of S[3] is set, setting the bit at position n + 256. The Rationale describes this as a scalability mechanism that lets a node receive only the messages it cares about when traffic is excessive.
Whisper envelope format
An envelope is an RLP-encoded structure of the form [ Expiry, TTL, Topic, Data, Nonce ]. Expiry is a 4-byte UNIX timestamp (seconds), TTL is a 4-byte time-to-live (seconds), Topic is 4 bytes of arbitrary data, Data is an arbitrary-length byte array holding the encrypted message, and Nonce is 8 bytes of arbitrary data used in the PoW computation.
Data field format, signatures, and encryption (optional)
The internal format of the Data field is optional and only matters if you intend to decrypt the incoming message. Messages in other formats are also valid and must be relayed to peers as-is. The plaintext concatenates flags (1 byte, where the first two bits give the auxiliary field size and the third bit indicates whether a signature is present), the auxiliary field (up to 4 bytes, the payload size), payload, padding, and signature (65 bytes if present) in that order, and if symmetric encryption is used a 12-byte salt (AES nonce) is appended. The signature is an ECDSA signature over the Keccak-256 hash of the data prior to encryption, serialized as R, S, V, where V must be 27 or 28. A party that cannot decrypt the message also cannot access the signature. Padding exists to prevent significant metadata from being revealed by message size alone, and it is recommended to pad the Data field (excluding the salt) to a multiple of 256 bytes before encryption. Encryption uses standard ECIES over a SECP-256k1 public key for asymmetric encryption, and AES GCM with a random 96-bit nonce for symmetric encryption.
P2P Request and P2P Message
P2P Request (0x7E) and P2P Message (0x7F) each carry a single Whisper envelope. P2P Request is used for Dapp-level peer-to-peer requests, such as a Whisper mail client asking a mail server for past messages. P2P Message is for peer-to-peer messages that must not be forwarded to other nodes, and is used, for example, when a mail server delivers expired past messages. According to the Rationale, without a P2P message, an expired message would be treated as a protocol violation and cause the peer connection to be dropped, making it impossible to deliver past messages; this can also be useful for cases where spending time on PoW is undesirable, such as an exchange's real-time trade feed.
Compatibility and implementation status
This EIP is compatible with Whisper version 6. A client that does not implement a given packet code must silently ignore packets with that code, which ensures forward compatibility. As of the time of writing, the golang implementation (v6) already uses codes 0x00 through 0x03, and Parity's v6 implementation is expected to use the same codes. 0x7E and 0x7F are only reserved and not yet used, left available for custom implementations of Whisper mail servers. The document status is Final, meaning the specification is settled.