Imagine a postal service where every letter is sealed with a lock that only the recipient can open - and you, the postal worker, never have a key. That’s end-to-end encryption: the server routes millions of sealed envelopes per second without being able to read any of them. Now add the requirement that letters addressed to someone currently away from home must be held safely, delivered exactly once when they return, and expire after 30 days if they never come back.

That’s the WhatsApp delivery problem. At 100 billion messages per day across 2+ billion users, the scale is staggering - but the real engineering challenge is not throughput, it’s the combination of guarantees that must hold simultaneously. Messages must arrive in order. Each must be delivered exactly once. End-to-end encryption must remain intact across the routing layer. Offline users can reconnect days later and must receive all queued messages in the right sequence. And push notifications must fire for the offline case without creating a delivery race condition with the WebSocket reconnect.

A naive approach - store messages in a database, poll for new ones - fails on latency (polling at 100ms intervals means 100ms average delay, not 100ms maximum), consistency (two concurrent poll queries might deliver the same message twice), and encryption (storing decryptable ciphertext on a central server breaks E2E guarantees). A more sophisticated attempt - a central message queue per user - fixes latency but breaks under multi-device scenarios: if the same user is connected on both phone and laptop, which device drains the queue?

The forces in tension are: delivery latency vs. durability of the offline queue, exactly-once delivery vs. horizontal scaling of delivery servers, multi-device fan-out vs. the single-recipient model of E2E encryption, and push notification timing vs. the ACK-based delivery state machine. We need to solve for end-to-end encryption key management, the message queue per recipient device, the ACK protocol, push notification fallback, and re-delivery on reconnect.

Requirements and Constraints

Functional Requirements

• Send a message from sender to recipient in under 100ms when both are online
• Queue messages durably when the recipient is offline (up to 30 days)
• Deliver all queued messages in order when the recipient reconnects
• Support multi-device: a user’s messages arrive on all their registered devices
• Provide delivery receipts: sent, delivered, read
• Support message expiry: configurable self-destruct timers (30s to 7 days)
• End-to-end encrypt every message - the server must never see plaintext

Non-Functional Requirements

• 100 billion messages per day = ~1.16 million messages/second average, ~4M/second at peak
• Online delivery latency: < 100ms p99
• Offline queue durability: messages survive server crashes, available after reconnect
• Queue depth: up to 10,000 messages per device during offline period
• Offline message retention: 30 days, then expired and sender notified
• Availability: 99.99% (< 52 minutes downtime per year)
• Concurrent WebSocket connections: 2 billion sustained

Constraints

• Encryption keys are managed by clients; the server stores only public keys
• Messages contain only encrypted ciphertext from the server’s perspective
• Each device has its own set of encryption keys (Signal Protocol)
• Multi-device fan-out happens at the sender client, not the server

High-Level Architecture

The system has six major components: a Connection Layer that maintains WebSocket connections for online users, a Message Router that dispatches inbound messages to the right delivery path, an Offline Queue that durably stores messages for disconnected devices, a Key Distribution Service that serves public keys for encryption, an ACK Protocol Engine that tracks delivery state, and a Push Notification Gateway that wakes offline devices.

Data flows in two distinct paths. Online path: sender client sends an encrypted message to the connection server via WebSocket; the connection server forwards to the Message Router; the router finds the recipient’s active connection server and delivers directly via server-to-server RPC; the recipient’s connection server pushes the message down the WebSocket; the recipient client sends an ACK; the ACK propagates back to the sender. Total: < 100ms. Offline path: Message Router finds no active connection for the recipient’s device; message is written to the Offline Queue with a 30-day TTL; Push Notification Gateway fires a silent push to wake the device; when the device reconnects, it drains its offline queue in order.

The Connection Layer

The Connection Layer maintains long-lived WebSocket connections for 2 billion concurrent users. Each connection server (a stateful process) holds hundreds of thousands of WebSocket connections. A global registry maps {user_id, device_id} to the connection server handling that device’s current session.

The connection registry is a distributed in-memory store (backed by Cassandra for durability). Each entry has a TTL of 90 seconds, refreshed by a client heartbeat every 30 seconds. If a device fails to send a heartbeat for 90 seconds, its registry entry expires and it is considered offline.