Skip to content

Backpressure#

Problem statement (interviewer prompt)

A fast producer feeds into a slower consumer through a queue. Without intervention, the queue grows without bound and the system OOMs. Design a flow-control mechanism so the producer slows down or drops gracefully when the consumer falls behind.

Backpressure is the explicit signal from a downstream consumer to an upstream producer that says "slow down". It replaces unbounded queues with bounded ones that push back via blocking, dropping, or credit acknowledgements.

flowchart LR
  P([Producer]) -->|push event| Q[(Bounded queue<br/>capacity 1000)]
  Q -->|pull| C([Consumer])
  Q -. queue full .-> P
  P -. wait / drop / slow .-> P

    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    class P,C service;
    class Q queue;

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;

Reactive Streams, gRPC flow control, TCP's sliding window, and Kafka consumer lag are all backpressure mechanisms at different layers.

Backpressure is the explicit signal that a consumer is the bottleneck, and the producer must adapt. Without it, fast producers turn every middle layer into an unbounded queue, then a crash.

The four responses#

When a consumer falls behind, producers can do one of four things:

Response Behaviour Example
Block Producer pauses until consumer drains Blocking queue, TCP write
Buffer Bounded queue; capacity is the safety net Kafka producer linger, Disruptor
Drop Newest or oldest discarded UDP, sampling logs, ad clicks
Reroute / shed Send elsewhere or 429 to client Rate limiter, sidecar bulkhead

Push vs pull#

flowchart TB
  subgraph Push[Push: producer drives]
    P1[Producer] -->|emit| C1[Consumer]
    C1 -. overflow? .-> X[no signal]
  end
  subgraph Pull[Pull: consumer drives]
    P2[Producer] -.holds.-> Buf[(buffer)]
    C2[Consumer] -->|request n| Buf
    Buf --> C2
  end

    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    class P1,P2,C1,C2 service;
    class Buf queue;

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;

Pull-based pipelines have backpressure built in - the consumer asks for n items at its own pace. Push-based pipelines need an out-of-band signal.

Credit-based flow control#

Each consumer hands out credits; each credit lets the producer send one message. Used in gRPC HTTP/2, RSocket, AMQP.

sequenceDiagram
  participant P as Producer
  participant C as Consumer
  C-->>P: window=100 credits
  loop while credits>0
    P->>C: msg (credits--)
  end
  Note over C: process 50
  C-->>P: window+=50

Reactive Streams contract#

Subscriber.onSubscribe(s) -> s.request(n)
Publisher.next(item)      -> repeat until n exhausted
Subscriber.request(more)  -> producer resumes

Implementations: Project Reactor (Flux), RxJava, Akka Streams, Java Flow.Subscriber. The single rule: nothing flows unless request(n) was called.

Layer-by-layer examples#

Layer Mechanism
TCP Sliding receive window
HTTP/2 / gRPC Per-stream WINDOW_UPDATE frames
Kafka Consumer lag triggers producer throttle
JMS / RabbitMQ Prefetch count
Disruptor / LMAX Ring buffer with sequence barriers
Reactor / RxJava onBackpressureBuffer / Drop / Latest
Database driver Connection pool acquire timeout

Shedding strategies#

When buffering is exhausted, shed:

  • Random load shedding: drop x% uniformly.
  • Priority-based: keep premium traffic, drop free tier.
  • Adaptive concurrency: Vegas/Gradient algorithms shrink in-flight limit on RTT spikes.

Common mistakes#

  • Unbounded queues (new LinkedList()) - the bug is "no backpressure".
  • Hiding pressure by autoscaling the consumer: cost balloons.
  • Backpressure into a queue that other tenants share: head-of-line blocking.
  • Treating retries as free: a retry storm is a producer that ignored backpressure.

Where backpressure shows up#

flowchart TB
  BP((Backpressure))
  RES[Resilience patterns<br/>bulkhead, circuit breaker]
  IDR[Idempotency / retries<br/>safe retry without flood]
  PS[Pub/Sub<br/>broker-mediated]
  STR[Batch / stream processing<br/>windowed flow]
  RES --> BP
  IDR --> BP
  BP --> PS
  BP --> STR

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;
    class BP service;
    class RES,IDR,PS,STR datastore;

Glossary & fundamentals#

Tag Concept What it is Page
HLD Resilience patterns timeouts, bulkheads, circuit breakers resilience-patterns
HLD Idempotency retries safe retry behaviour under pressure idempotency-retries
HLD Pub Sub Pattern broker-mediated backpressure pub-sub-pattern
HLD Batch stream processing windowed flow with bounded state batch-stream-processing

Quick reference#

Four responses when overloaded#

Response Use
Block Producer can wait; durability matters
Buffer Burst-smoothing with bounded capacity
Drop Sampling, telemetry, UDP
Shed Return 429 / 503 to client

Push vs pull#

  • Push: producer drives; needs explicit signal (credits, windows).
  • Pull: consumer drives; backpressure is implicit in request(n).

Reactive Streams (one rule)#

Subscriber.request(n)  -> Publisher may emit up to n items

Per-layer mechanism#

Layer Mechanism
TCP Receive window
HTTP/2 WINDOW_UPDATE frames
Kafka Consumer lag throttles producer
RabbitMQ Prefetch count
Reactor onBackpressureBuffer/Drop/Latest

Shedding strategies#

  • Random drop x%
  • Priority queues
  • Adaptive concurrency (Gradient, Vegas)

Anti-patterns#

  • Unbounded new LinkedList()
  • Auto-scaling away the symptom
  • Sharing a queue across tenants (head-of-line)
  • Retry storms with no jitter

Refs#

  • Reactive Streams spec
  • SEDA paper
  • Netflix - Performance under load

FAQ#

What is backpressure in software systems?#

Backpressure is the explicit signal from a slow consumer to a fast producer telling it to slow down, drop, or wait, so queues stay bounded and the system avoids running out of memory.

What is the difference between backpressure and rate limiting?#

Rate limiting caps how many requests a producer can send based on a static policy. Backpressure is dynamic feedback from the consumer itself, reacting to real-time load and queue depth.

How does backpressure work in reactive streams?#

The consumer issues a demand signal (request N items). The producer can only emit up to N items, so the consumer always controls flow. Project Reactor, RxJava, and Akka Streams all use this contract.

What happens without backpressure?#

Unbounded queues grow until memory is exhausted, GC pauses spike, and the process eventually OOMs. Latency goes through the roof long before that, because requests pile up waiting in the queue.

How is backpressure implemented in Kafka?#

Kafka uses a pull model where consumers ask for batches of records. If the consumer is slow, lag grows on the broker but the producer continues. Quotas and pause/resume APIs add explicit control.

Further reading#