Sticky Sessions

Problem statement (interviewer prompt)

A web service stores per-user session state (cart, draft, partial upload progress) in memory on each backend. Behind a round-robin load balancer, subsequent requests land on different backends and lose state. Design the routing layer so a given client always lands on the same backend, without making backends a single point of failure.

Sticky (or "session affinity") routing makes the load balancer remember which backend handled a client and forward all subsequent requests there. Two common implementations: source-IP hashing and cookie-based affinity.

flowchart LR
  C1([Client A]) --> LB[Load balancer]
  C2([Client B]) --> LB
  LB -->|hash A -> 1| B1[Backend 1]
  LB -->|hash B -> 2| B2[Backend 2]
  LB --> B3[Backend 3]
  B1 -. session A in-memory .-> S1[(session)]
  B2 -. session B in-memory .-> S2[(session)]

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    class C1,C2 client;
    class LB edge;
    class B1,B2,B3 service;
    class S1,S2 cache;

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;

Stateless backends with an external session store (Redis, JWT) are the modern default; stickiness is the pragmatic fallback when migration is too costly.

Session affinity solves the simplest version of "stateful clients on a horizontal fleet" without requiring you to externalise state. It also introduces failure modes that quiet down with stateless designs.

Three implementation strategies

1. Source-IP hashing

flowchart TB
  C([Client 1.2.3.4]) --> LB[L4 LB]
  LB -.->|hash 1.2.3.4| Pick[backend index 1]
  LB --> B1[Backend 1]
  LB --> B2[Backend 2]
  LB --> B3[Backend 3]

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;

• Pros: works at L4, no cookies, supports any protocol.
• Cons: NAT collapses many users to one IP; mobile IPs change; rebalances on backend add/remove.

2. Cookie-based (LB-set or app-set)

The LB writes SERVERID=b2 on first response; subsequent requests echo it back. The LB inspects the cookie and routes accordingly.

Cookie source	Example
LB-set	NGINX sticky cookie, AWS ALB AWSALB
App-set	App writes JSESSIONID, LB hashes the value

• Pros: per-user precision, NAT-safe.
• Cons: requires L7 LB, breaks for non-HTTP traffic, leaks server identity.

3. Consistent-hash ring

The LB hashes the cookie or IP onto a consistent ring of backends. Adding/removing a backend reshuffles only K/N sessions instead of all.

Failure handling

sequenceDiagram
  participant C as Client
  participant LB
  participant B2 as Backend 2 (sticky)
  participant B3 as Backend 3
  C->>LB: req w/ cookie SERVERID=b2
  LB->>B2: forward
  Note over B2: backend dies
  LB-xB2: TCP reset
  LB->>B3: failover
  B3-->>C: lost in-memory session -> 200 OK but no draft

Stickiness alone doesn't survive backend failure. Pair it with: - An external session store (Redis) for recoverable state. - Idempotent operations so re-routing is safe.

Where stickiness is unavoidable

Workload	Why
WebSocket / SSE long connections	The TCP connection is pinned for its lifetime
In-memory game state	Migration cost too high per tick
Stateful uploads (multipart, resumable)	Chunks must hit the host holding the temp file
Legacy J2EE / HttpSession	Application uses in-memory session by default

Migrating to stateless

flowchart TB
  subgraph Before[Before: sticky]
    LB1[LB] --> S1[Server 1<br/>HttpSession]
    LB1 --> S2[Server 2<br/>HttpSession]
  end
  subgraph After[After: stateless]
    LB2[LB] --> S3[Server 1]
    LB2 --> S4[Server 2]
    S3 --> R[(Redis / JWT)]
    S4 --> R
  end
  Before -.refactor.-> After

    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    class LB1,LB2 edge;
    class S1,S2,S3,S4 service;
    class R cache;

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;

• Server-side: move session to Redis/Memcached with a TTL. Backends become identical and interchangeable.
• Client-side: serialize session into a signed JWT cookie. No server-side storage but cookie size grows and revocation is harder.

Where stickiness fits

flowchart TB
  SS((Sticky<br/>sessions))
  LB[Load balancer<br/>hosts the policy]
  RT[Realtime protocols<br/>WebSocket needs pin]
  CACHE[Caching strategies<br/>external session store]
  CH[Consistent hashing<br/>cookie-based affinity]
  LB --> SS
  SS --> RT
  SS --> CACHE
  CH -. cookie hash .-> SS

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;
    class SS service;
    class LB,RT,CACHE,CH datastore;

Glossary & fundamentals

Tag	Concept	What it is	Page
HLD	Load balancer	the layer that implements affinity	load-balancer
HLD	Caching strategies	external session store as a cache	caching-strategies
HLD	Consistent hashing	ring algorithm for cookie-based stickiness	consistent-hashing
HLD	Realtime protocols	WebSocket pinning context	realtime-protocols

Quick reference

Why

Pin a client to a single backend so in-memory session state stays accessible across requests.

Implementations

Approach	LB layer	Pros	Cons
Source-IP hash	L4	Protocol-agnostic	NAT collapse, mobile churn
LB-set cookie	L7	Per-user accuracy	HTTP-only
App-set cookie	L7	App controls identity	Coordination needed
Consistent-hash on cookie	L7	Minimal rebalance	Slightly more complex

When to use stickiness

• WebSocket / SSE connections
• In-memory game state
• Resumable / multipart uploads
• Legacy HttpSession apps

When to avoid

• Stateless REST services
• Microservices with horizontal autoscaling
• Anywhere a backend failure causes user-visible loss

Recipe for stateless backends

• Externalise session to Redis with TTL, or
• Sign session payload into a JWT cookie

Failure modes

• Backend death → sticky route fails over → in-memory session lost
• Cookie tampering → users land on wrong backend (mitigate with HMAC)
• Backend add/remove → all keys rehash unless consistent hashing used

Refs

• NGINX, AWS ALB, Cloudflare load-balancing docs

FAQ

What are sticky sessions?

A load balancer setting that routes every request from one client to the same backend. It is implemented via a cookie set by the LB or by hashing the source IP address.

When do I need sticky sessions?

When backends keep per-user state in memory, like a draft form, partial upload, or a WebSocket connection. Stateless services with external session stores do not need affinity.

What is the downside of sticky sessions?

Traffic distribution gets uneven, backend restarts disconnect users, and scaling out is slower because new pods take time to attract sessions. Failure of one backend loses its sessions.

Should I store sessions externally instead?

Yes for most modern apps. Putting session state in Redis or a token like JWT lets any backend handle any request, simplifying scaling, deployments, and zero-downtime restarts.

How does cookie-based affinity work?

The load balancer sets a cookie identifying which backend handled the first request. Subsequent requests carry the cookie and the LB forwards them to the same instance.

Related Topics

• Load Balancer: the host of every affinity policy described here
• Caching Strategies: externalising session state is a write-through cache pattern
• Realtime Protocols: WebSocket and SSE inherently need stickiness for their lifetime

Further reading

• Doc - NGINX - sticky module
• Doc - AWS ALB - sticky sessions
• Blog - Cloudflare - Load balancing session affinity