Real-time Protocols#

Q: When should I use WebSocket instead of SSE?

Use WebSocket when you need bidirectional, low-latency messaging like chat, multiplayer games, or collaborative editing. Use SSE when the server only pushes to the client.

Q: How many WebSocket connections can one server handle?

A well-tuned gateway holds 50k to 200k concurrent connections with about 10 KB of memory per idle socket. Kernel file descriptor and TCP settings matter more than CPU.

Q: Why do real-time apps still fall back to long polling?

Some corporate proxies and old firewalls block the WebSocket upgrade. Long polling uses plain HTTP, so it traverses any network that allows normal web traffic.

Q: How do you scale WebSocket across many servers?

Use sticky load balancing to keep a client on one gateway and a pub/sub bus like Redis, NATS, or Kafka to fan events out to every gateway holding interested subscribers.

Q: What heartbeat interval should WebSocket use?

Send a ping frame every 20 to 30 seconds and close the connection if no pong arrives within two intervals. This survives idle timeouts on most L4 load balancers.

Problem statement (interviewer prompt)

Design the transport for a real-time product (chat / live scores / collaborative editor). Compare WebSocket, SSE, long polling and gRPC streaming. Pick one with reasoning, and explain how you'd scale to 10M concurrent connections.

flowchart LR
  C([Client])
  P[Long Polling<br/>open until event]
  S[SSE<br/>server -> client only]
  W[WebSocket<br/>bidirectional]
  G[gRPC streaming<br/>4 variants]
  R[Server]
  C --> P --> R
  C --> S --> R
  C --> W --> R
  C --> G --> R

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;
    class C,S client;
    class P,W,G,R service;

Four ways to push from server to client: long polling (HTTP request held open), SSE (server-sent events, one-way), WebSocket (full-duplex), gRPC streaming (HTTP/2 server / client / bidi streams).

sequenceDiagram
  participant C as Client
  participant S as Server

  Note over C,S: Long Polling
  C->>S: GET /events (held open)
  Note over S: event happens
  S-->>C: 200 response with event
  C->>S: GET /events (re-poll immediately)

  Note over C,S: SSE
  C->>S: GET /events  (Accept: text/event-stream)
  S-->>C: data: event 1\n\n
  S-->>C: data: event 2\n\n
  Note over S: same connection forever

  Note over C,S: WebSocket
  C->>S: HTTP Upgrade: websocket
  S-->>C: 101 Switching Protocols
  C->>S: frame
  S->>C: frame
  Note over C,S: full duplex

Comparison#

	Long polling	SSE	WebSocket	gRPC streaming
Direction	server → client (1 event/conn)	server → client	bidirectional	uni or bidi
Transport	HTTP/1.1	HTTP/1.1 + 2	TCP after upgrade	HTTP/2
Re-connect	client polls	EventSource auto-reconnects	manual	manual
Binary	base64 or no	no (text only)	yes	yes (protobuf)
Proxy / firewall friendly	best	very good	good (port 443 + Upgrade)	varies
Backpressure	implicit (next poll)	none	application-level	flow control built-in
Use case	legacy clients, simple pushes	dashboards, news feeds, notifications	chat, collab, games	service-to-service streams

Connection model#

flowchart TB
  subgraph Client
    A[App]
    SDK[WS / SSE / gRPC SDK]
  end
  subgraph Edge
    LB[L7 LB / proxy]
    GW[Gateway holding persistent conns]
  end
  subgraph Backend
    H[Hub / Channel Service]
    PB[Pub/Sub bus]
    SVC[Domain services]
  end
  A --> SDK --> LB --> GW
  GW --> H
  H --- PB
  SVC --> PB
  PB --> H --> GW --> SDK

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;
    class SDK client;
    class LB,GW edge;
    class A,H,SVC service;
    class PB queue;

Scale numbers#

Single WS gateway: 50k-200k concurrent connections (kernel tuning matters).
SSE same range; less per-connection memory.
Long polling far worse - thread/conn churn unless using async stack.

Choosing one#

One-way push, browser-friendly, simple → SSE.
Bidirectional, low-latency, chat-like → WebSocket.
Service-to-service → gRPC streaming.
You need to traverse every legacy proxy → long polling fallback.

Many systems (Slack, ChatGPT, Discord) use WebSocket as primary + long-polling fallback for restrictive networks.

Operational pitfalls#

Sticky load balancer or message bus: each WS / SSE connection is pinned; cluster needs pub/sub fan-out (Redis pub/sub, NATS, Kafka).
Idle timeouts: L4 LBs (AWS NLB) close idle TCP at 60-350 s - send heartbeats.
Server reboot / deploy: thundering reconnect when 200k clients reconnect simultaneously - use jitter + staged rollouts.
Auth on upgrade: pass a short-lived token in the initial handshake; rotate.

Glossary & fundamentals#

Concepts referenced in this design. Each row links to its canonical page; the tag column shows whether it is a high-level (HLD) or low-level (LLD) concept.

Tag	Concept	What it is	Page
`HLD`	Load balancer / GSLB	L4/L7 traffic distribution and failover	load-balancer
`HLD`	Pub/Sub & message brokers	topics, consumer groups, delivery semantics	pub-sub-pattern
`HLD`	Idempotency & retries	safe re-execution, backoff + jitter	idempotency-retries
`HLD`	Resilience patterns	timeout, retry, breaker, bulkhead, backpressure	resilience-patterns
`HLD`	HTTP / TLS protocols	HTTP 1.1/2/3, QUIC, TLS 1.3	http-protocols
`HLD`	Realtime protocols	WS / SSE / polling / gRPC streaming	realtime-protocols
`LLD`	Async models	futures / async-await / coroutines / actors	async-models
`LLD`	Structural patterns	Adapter, Decorator, Facade, Proxy, Composite	structural-patterns

Quick reference#

Heartbeat / keep-alive#

WS: send ping frame every 20-30 s; close conn if no pong in 2× interval.
SSE: send : comment\n\n every 15 s.
Long-poll: set server response timeout < intermediary idle timeout.

Backpressure#

WS / SSE have no built-in flow control - application must drop or coalesce on slow clients.
gRPC streaming uses HTTP/2 flow control (per-stream + connection windows).

Auth at the edge#

Use cookies or Authorization header for HTTP-based (SSE).
For WS, browsers can't set custom headers on the upgrade - pass auth as query param or sub-protocol, or do an HTTP login first that sets a cookie.

Capacity#

Plan ~10 KB RSS per idle WS connection.
200k conns × 10 KB = 2 GB per gateway box; tune fs.file-max, net.ipv4.tcp_*.

Refs#

RFC 6455 (WebSocket), WHATWG SSE spec.
"Designing for Throughput" - Cloudflare Workers WS blog.
Slack / Discord / Phoenix Channels architecture posts.

FAQ#

When should I use WebSocket instead of SSE?#