WhatsApp / Messenger#

Q: How does WhatsApp deliver messages in real time?

Each client holds a long-lived WebSocket connection to a gateway. Messages are routed to a per-user inbox queue and pushed instantly if the recipient is online, or via APNS/FCM when offline.

Q: What encryption does WhatsApp use end to end?

WhatsApp uses the Signal Protocol with X3DH for key agreement and the Double Ratchet for per-message forward secrecy, encrypting each message per recipient device.

Q: How does WhatsApp handle group chats at scale?

Groups use Sender Keys from the Signal protocol so the sender encrypts once per device, and the server fans out copies to each member rather than the sender re-encrypting per recipient.

Q: Where does WhatsApp store undelivered messages?

Undelivered messages sit in a per-user inbox queue backed by Cassandra or Mnesia. WhatsApp deletes messages from the server once they are delivered to preserve end to end privacy.

Q: How does WhatsApp scale to billions of users?

WhatsApp historically ran on Erlang on the BEAM VM using an actor-per-user model, with sharded WebSocket gateways pinned to user sessions and roughly 100k connections per box.

Problem statement (interviewer prompt)

Design WhatsApp/Messenger: 1:1 + group chat with E2E encryption, presence + typing + read receipts, voice/video calls, multi-device sync, and push notifications when offline. Handle 100B+ messages/day and 2B users.

flowchart LR
  A([User A])
  B([User B])
  WS[WebSocket Gateway]
  CH[Chat Service]
  MQ[(Message Inbox)]
  PRES[Presence]
  PUSH((APNS / FCM))
  A --> WS --> CH --> MQ
  CH --> WS
  WS --> B
  PRES --- WS
  CH -. offline .-> PUSH --> B

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;
    class A,B client;
    class WS edge;
    class CH,PRES service;
    class MQ datastore;
    class PUSH external;

flowchart TB
  subgraph Devices
    A([Phone A])
    B([Phone B])
    G[Group members ...]
    DESK([Desktop / Web - paired])
  end

  subgraph Edge
    DNS
    LB[Anycast L4]
    GW[Long-lived WebSocket Gateway<br/>per-user pinned]
  end

  subgraph Core[Core Services]
    AUTH[Auth / SIM verification]
    SESS([Session / Device registry])
    CHAT[Chat Service<br/>routing + storage]
    GRP[Group Service]
    PRES[Presence Service]
    TYP[Typing indicator]
    READ[Read-receipts]
    MED[Media Service]
    OBJ[(Encrypted blob store)]
    KEYS[Key Server<br/>E2E pre-keys, prekey bundles]
  end

  subgraph Queues
    INBOX[(Per-user inbox queue<br/>persisted Cassandra/Mnesia)]
    UNDLV[[Undelivered queue]]
  end

  subgraph PushTier
    APNS
    FCM
    HMS
  end

  subgraph Crypto[E2E - Signal Protocol]
    XX[X3DH key agreement]
    DR[Double Ratchet<br/>per-message forward secrecy]
    DEV[[Multi-device fan-out<br/>encrypt per device]]
  end

  subgraph Calls[Voice / Video]
    SIG[Signaling]
    SFU[SFU - Selective Forwarding]
    TURN[STUN / TURN servers]
  end

  subgraph Storage
    META[(Chat metadata)]
    HIST[(Encrypted chat history<br/>on device only)]
    BAK[Optional backup<br/>encrypted iCloud / GDrive]
  end

  subgraph Obs
    M[Metrics]
    LOG[Logs]
  end

  A --> DNS --> LB --> GW
  B --> LB
  GW --> CHAT
  CHAT --> INBOX
  CHAT --> GRP
  CHAT --> READ
  CHAT --> TYP
  CHAT --> MED --> OBJ
  CHAT -. recipient online .-> GW
  CHAT -. recipient offline .-> UNDLV --> PushTier
  PUSHTIER[(Push tier)] --> B
  AUTH --> SESS
  SESS --> GW
  KEYS --> Crypto
  Crypto --> A
  Crypto --> B
  PRES --- GW
  Calls --- GW
  Calls --- TURN
  CHAT --> META
  GRP --> META

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;
    class A,B,DESK,SESS client;
    class LB,GW edge;
    class G,AUTH,CHAT,GRP,PRES,TYP,READ,MED,KEYS,XX,DR,SIG,SFU,TURN,BAK service;
    class INBOX,META,HIST,PUSHTIER datastore;
    class UNDLV,DEV queue;
    class OBJ storage;
    class M,LOG obs;

Message lifecycle#

Sender encrypts payload per recipient device (E2E).
Send over WS to gateway → CHAT.
CHAT writes to recipient inbox queue, ack to sender ("sent" ✓).
If recipient online: push via WS, ack "delivered" ✓✓.
On read: client sends read receipt → "read" ✓✓ (blue).
Server purges from inbox once delivered (WhatsApp deletes after delivery for E2E).

Group chat#

Server fans out one copy per device (each E2E-encrypted with that device key).
Group state stored as membership list; messages keyed by (group_id, ts, msg_id).
Sender Keys (Signal protocol) for groups to avoid quadratic blow-up.

Presence#

Heartbeat / WS state; eventual consistency across regions.
Last-seen privacy controls.

Storage strategy#

Server holds messages only until delivered (WhatsApp), or full history (Messenger).
Device holds local SQLite db; cloud backup is encrypted and key-derived from PIN / phone.

Scale#

2B+ users, 100B+ messages/day.
WhatsApp famously ran on Erlang (BEAM) with small ops team - concurrency model designed for actor-per-user.

Glossary & fundamentals#

Concepts referenced in this design. Each row links to its canonical page; the tag column shows whether it is a high-level (HLD) or low-level (LLD) concept.

Tag	Concept	What it is	Page
`HLD`	Load balancer / GSLB	L4/L7 traffic distribution and failover	load-balancer
`HLD`	CAP / PACELC	C vs A under partition; L vs C otherwise	cap-pacelc
`HLD`	Observability	metrics, logs, traces, SLOs	observability
`HLD`	Realtime protocols	WS / SSE / polling / gRPC streaming	realtime-protocols
`LLD`	Behavioural patterns	Strategy, Observer, State, Command, Chain	behavioral-patterns

Quick reference#

Functional#

1:1 and group chat (text, voice notes, media, location).
Delivery + read receipts, typing, presence.
Voice + video calls.
Multi-device, E2E encryption.
Push notifications when offline.

Non-functional#

p99 send latency < 500 ms.
2B users; 100B msgs/day.
99.99% uptime; survive regional outage.

Capacity#

100B msgs/day → ~1.2M/s avg, 5M/s peak.
Avg msg + envelope ~1 KB → ~100 TB/day before E2E.
Long-lived WS connections: 2B / 100k per box = 20k+ gateway boxes.

API (simplified)#

WS up: hello + auth token + device id
WS msg: send(chat_id, payload, msg_id, prekey?)
WS evt: delivered(msg_id) / read(msg_id) / typing(chat_id)

Schema#

users(phone, id, public_key)
devices(user_id, device_id, push_token, key_bundle)
groups(id, [member_id...])
inbox(user_id, msg_id, payload, ts) Cassandra, TTL after delivery

Trade-offs#

Sticky WS gateway simplifies routing; failover requires re-connect.
Server-deletes-after-delivery (WhatsApp) vs full server history (Messenger): privacy vs sync convenience.
E2E prevents server-side moderation; flag-based reporting compensates.
Multi-device pairing: device tree (one master, others linked) vs full mesh.

Refs#

WhatsApp engineering talks (Erlang scaling stories).
Signal Protocol papers (X3DH, Double Ratchet).
Facebook Messenger architecture blog posts.
Alex Xu Vol 2 "Design a chat system."

FAQ#

How does WhatsApp deliver messages in real time?#