TCP Congestion Control#

TCP congestion control is the sender-side algorithm that decides how many unacknowledged bytes a connection may keep in flight. Without it, every TCP sender on the internet would blast packets at link speed, fill router queues, and trigger a 1986 style congestion collapse where throughput drops to near zero even though links are physically fast. This page covers the TCP congestion control system design intuition for Reno, CUBIC, and BBR and how each one manages the congestion window.

The one-paragraph mental model#

Each TCP sender keeps a number called cwnd (congestion window). It can have at most min(cwnd, rwnd) unacknowledged bytes in flight at any moment, where rwnd is the receiver's advertised window. Congestion control is the rulebook that grows cwnd when the network seems healthy and shrinks it when the network seems congested. Flow control is the receiver protecting its buffer. Congestion control is the sender protecting the network between them.

flowchart LR
  Sender([Sender]) --> Bottleneck{{Bottleneck link}}
  Bottleneck --> Receiver([Receiver])
  Receiver -. ACK feedback .-> Sender
  Sender --- Knob[cwnd controller]

  classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
  classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
  classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
  class Sender,Receiver client;
  class Bottleneck edge;
  class Knob service;

The controller turns ACK arrivals (or their absence) into adjustments of cwnd. That feedback loop is the whole game.

Why TCP needs congestion control#

Imagine 100 senders sharing a 1 Gbps link. If every sender just transmits as fast as their NIC allows, the router buffer fills, packets drop, every sender retransmits, queues grow more, and effective goodput collapses below 100 Mbps. This is the failure mode Van Jacobson described in 1988 after the early ARPANET froze for days. The fix was to make every TCP sender voluntarily slow down based on network signals, even though that is locally suboptimal. The result is a kind of distributed bandwidth-sharing protocol where fairness emerges from millions of independent decisions.

Reno: slow start, AIMD, fast recovery#

Reno is the classic loss-based scheme. It has four phases:

Slow start. cwnd doubles every RTT (one extra segment per ACK) until it hits a threshold ssthresh or a loss.
Congestion avoidance. cwnd grows by one segment per RTT, a linear ramp.
Fast retransmit. Three duplicate ACKs trigger an immediate retransmit without waiting for the RTO.
Fast recovery. Halve cwnd, set ssthresh = cwnd / 2, return to congestion avoidance.

This pattern, grow by 1 each RTT, halve on loss, is called AIMD: Additive Increase, Multiplicative Decrease. AIMD provably converges to fair sharing among competing flows.

flowchart TB
  SS[Slow Start<br/>cwnd doubles per RTT] -->|loss or ssthresh| CA[Congestion Avoidance<br/>cwnd += 1 per RTT]
  CA -->|3 dupacks| FR[Fast Recovery<br/>cwnd halves]
  FR --> CA
  CA -->|RTO timeout| SS

  classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
  classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
  class SS,CA service;
  class FR cache;

Drawn over time, cwnd traces a sawtooth: climb linearly, drop in half, climb again. It is ugly but it is fair.

CUBIC: scaling to long fat pipes#

Reno's linear growth wastes capacity on a 100 Gbps coast-to-coast link with 70 ms RTT. The window needs to be hundreds of thousands of segments and Reno would take hours to ramp back up after a single loss. CUBIC replaces the linear growth with a cubic curve that quickly returns close to the prior window after a drop, then probes more cautiously past it. CUBIC has been the Linux default since kernel 2.6.19 (2006). It is still loss-based: a packet drop is the signal to back off.

BBR: model-based, not loss-based#

In 2016 Google released BBR (Bottleneck Bandwidth and Round-trip propagation time). Instead of waiting for loss, BBR continuously estimates two numbers:

BtlBw: the bottleneck bandwidth of the path.
RTprop: the minimum (no-queue) round-trip time.

The ideal in-flight data is BtlBw * RTprop, the bandwidth-delay product. BBR paces packets to match BtlBw and keeps in-flight near the BDP, so the bottleneck stays full but queues stay empty. It ignores loss as a congestion signal, which is why it dramatically out-performs CUBIC on lossy wifi and cellular links where drops are not caused by congestion.

YouTube, Google.com, and Spotify backends run BBR by default. Cloudflare and Dropbox use it on selective edges. Netflix Open Connect uses BBR for long-haul fills.

ECN: a politer signal#

Modern routers can mark packets with Explicit Congestion Notification instead of dropping them. The receiver echoes the mark back; the sender treats it as a soft loss signal and halves the window without anything actually being lost. Datacenter TCP (DCTCP) and the new L4S architecture lean heavily on ECN to keep queues short.

Where each one wins#

Algorithm	Best fit	Loses on
Reno / NewReno	Compatibility, textbooks	Long fat networks
CUBIC	General internet, Linux default	Lossy wireless, deep queues
BBR	Long fat networks, wifi, cellular	Shallow buffers, BBRv1 fairness
DCTCP	Datacenter, ECN-enabled fabric	Public internet

Picking the right algorithm is a one-line Linux tunable: sysctl -w net.ipv4.tcp_congestion_control=bbr. It applies per kernel, not per app.

The takeaway: congestion control is a small piece of code that decides whether your video starts in 200 ms or 2 seconds. Reno taught the internet to share. CUBIC taught it to scale. BBR taught it to ignore noise. All three still live in production kernels somewhere right now.

TCP congestion control is the part of the TCP stack that decides how much to send, in contrast to flow control which decides when the receiver can absorb more. It is the answer to a 1980s internet failure mode and remains an active research area in 2026. This guide covers Reno, CUBIC, and BBR; how the congestion window is managed; ECN; and the practical knobs you actually touch in production.

Why TCP needs congestion control#

In October 1986 the link between LBL and UC Berkeley dropped from a nominal 32 kbps to 40 bps, a thousand-fold degradation, even though the wires were physically fine. The cause was congestion collapse: every sender retransmitting on timeout, queues overflowing, and the network spending all of its capacity on duplicate packets. Van Jacobson's 1988 paper Congestion Avoidance and Control introduced slow start, congestion avoidance, fast retransmit, and fast recovery, the four mechanics that still anchor every loss-based scheme today.

The key distinction interviews probe:

Mechanism	Protects	Signal
Flow control	Receiver buffer	Advertised window (`rwnd`)
Congestion control	Network queues	Loss, RTT change, or ECN

A sender never has more than min(cwnd, rwnd) unacknowledged bytes in flight. Tune cwnd too low and you waste bandwidth; tune it too high and the bottleneck router drops packets, which hurts every flow sharing the link.

The classic Reno state machine#

Reno is the schoolbook scheme. Its state machine has four phases driven entirely by ACKs and timeouts:

stateDiagram-v2
  [*] --> SlowStart
  SlowStart --> CongestionAvoidance : cwnd >= ssthresh
  SlowStart --> FastRecovery : 3 dupacks
  CongestionAvoidance --> FastRecovery : 3 dupacks
  CongestionAvoidance --> SlowStart : RTO timeout
  FastRecovery --> CongestionAvoidance : new ACK
  FastRecovery --> SlowStart : RTO timeout

Slow start. cwnd begins at the initial window, set to 10 MSS by RFC 6928 (older kernels used 1 to 4). For every ACK, cwnd += 1, which effectively doubles it each RTT. The doubling continues until cwnd >= ssthresh (slow-start threshold) or a loss is detected.

Congestion avoidance. Linear growth: cwnd += 1 / cwnd per ACK, which yields one extra segment per RTT. This is the Additive Increase half of AIMD.

Fast retransmit + fast recovery. When the sender sees 3 duplicate ACKs for the same sequence number, it concludes a single packet was lost without waiting for the retransmission timeout. It retransmits immediately, sets ssthresh = cwnd / 2, sets cwnd = ssthresh, and continues in congestion avoidance. This is the Multiplicative Decrease half.

RTO timeout. If the retransmission timer fires before any new ACKs arrive, the network is presumed badly congested. ssthresh = cwnd / 2, cwnd = 1 MSS, and the sender drops back to slow start.

Drawn against time, cwnd traces the famous sawtooth:

flowchart LR
  T0[t = 0<br/>cwnd small] --> T1[slow start<br/>doubles per RTT]
  T1 --> T2[ssthresh<br/>switch to AIMD]
  T2 --> T3[linear growth]
  T3 --> T4[loss event<br/>cwnd halves]
  T4 --> T3

  classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
  classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
  class T0,T1,T2,T3 service;
  class T4 cache;

Pseudocode for Reno#

# Sender-side congestion controller, NewReno style
cwnd = 10 * MSS          # initial window per RFC 6928
ssthresh = 64 * 1024     # large initial threshold
state = "slow_start"
dupack_count = 0

def on_new_ack(acked_bytes):
    global cwnd, ssthresh, state, dupack_count
    dupack_count = 0
    if state == "slow_start":
        cwnd += MSS                       # +1 per ACK = doubles per RTT
        if cwnd >= ssthresh:
            state = "cong_avoid"
    elif state == "cong_avoid":
        cwnd += MSS * MSS / cwnd          # +1 segment per RTT
    elif state == "fast_recovery":
        state = "cong_avoid"
        cwnd = ssthresh

def on_duplicate_ack():
    global dupack_count, cwnd, ssthresh, state
    dupack_count += 1
    if dupack_count == 3 and state != "fast_recovery":
        ssthresh = max(cwnd // 2, 2 * MSS)
        cwnd = ssthresh + 3 * MSS         # inflate for in-flight dupacks
        state = "fast_recovery"
        retransmit_oldest_unacked()

def on_rto_timeout():
    global cwnd, ssthresh, state
    ssthresh = max(cwnd // 2, 2 * MSS)
    cwnd = MSS
    state = "slow_start"
    retransmit_oldest_unacked()

This 30-line skeleton is structurally the same as the production code in the Linux kernel at net/ipv4/tcp_cong.c plus the per-algorithm modules.

Why Reno is not enough: the long fat pipe problem#

Consider a 10 Gbps coast-to-coast link, RTT 70 ms. The bandwidth-delay product is 10 Gbps * 70 ms = 87.5 MB, roughly 60000 MSS-sized segments. After a single packet drop, Reno cuts cwnd to 30000 and rebuilds at +1 per RTT. To get back to 60000 takes 30000 RTTs, which is 35 minutes. In 35 minutes, the long-haul TCP flow runs at half capacity. CUBIC fixes exactly this.

CUBIC: cubic window growth#

CUBIC, introduced in 2008 and the Linux default since kernel 2.6.19, changes only the growth rule. The window after a drop is a cubic function of time since the drop:

W(t) = C * (t - K)^3 + W_max
where:
  W_max = cwnd at the drop
  K = cube_root( W_max * (1 - beta) / C )
  beta = 0.7   # CUBIC's multiplicative decrease factor
  C = 0.4      # CUBIC scaling constant

The shape: cwnd climbs steeply right after the drop, flattens as it approaches the previous max, plateaus near W_max (the concave region, polite probing), then ramps steeply past it (the convex region, aggressive probing for new capacity).

flowchart TB
  Drop[Loss event<br/>cwnd halves to 0.7 * W_max] --> Concave[Concave region<br/>fast return to W_max]
  Concave --> Plateau[Plateau<br/>cautious probing]
  Plateau --> Convex[Convex region<br/>aggressive probing]
  Convex --> NewDrop[Next loss event]

  classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
  classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
  classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
  class Drop cache;
  class Concave,Plateau,Convex compute;
  class NewDrop cache;

CUBIC is RTT-independent for its growth, so flows with different RTTs share bandwidth more fairly than they would under Reno (Reno favors short-RTT flows). It is still loss-based though; an unrelated wifi drop will halve the window even when the path is uncongested.

BBR: model-based congestion control#

In 2016 Google published BBR (Bottleneck Bandwidth and Round-trip propagation time). BBR ditches the loss-based paradigm. Instead it continuously estimates:

BtlBw: the bottleneck bandwidth, sampled as the highest delivery rate observed.
RTprop: the minimum RTT, sampled as the lowest RTT observed in the last 10 seconds.

The product BtlBw * RTprop is the bandwidth-delay product, the optimal amount of data in flight. BBR's cwnd is roughly 2 * BDP (to absorb ACK clocking) and packets are paced at BtlBw, not bursted.

flowchart TB
  subgraph BBR_State_Machine[BBR phases]
    direction TB
    Startup[Startup<br/>exponential probe] --> Drain[Drain<br/>flush queue]
    Drain --> ProbeBW[ProbeBW<br/>cycle pacing gain]
    ProbeBW --> ProbeRTT[ProbeRTT<br/>cwnd = 4 segments]
    ProbeRTT --> ProbeBW
  end

  classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
  classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
  class Startup,Drain service;
  class ProbeBW,ProbeRTT compute;

Startup. Like slow start but driven by bandwidth: increase the pacing gain to 2 / ln(2) until measured bandwidth stops growing.

Drain. Briefly pace below BtlBw to drain the queue built up in startup.

ProbeBW. Steady-state cycling through pacing gains [5/4, 3/4, 1, 1, 1, 1, 1, 1]. The 5/4 cycle probes for more bandwidth; the 3/4 cycle drains the resulting queue.

ProbeRTT. Every 10 seconds drop cwnd to 4 segments for 200 ms to re-measure RTprop. Otherwise the min-RTT estimate drifts upward if a queue builds.

Why BBR matters#

Lossy wireless. Cellular and wifi lose packets to interference, not congestion. CUBIC cuts the window on every drop and stalls. BBR ignores loss and keeps the pipe full.
Long fat pipes. Coast-to-coast and intercontinental flows hit full bandwidth in a few RTTs instead of minutes.
Lower tail latency. Because BBR keeps queue depth near zero at the bottleneck, request latency through a congested router is far lower than with CUBIC, which deliberately fills queues until they overflow.

YouTube measured a 4 percent throughput gain on average and 14 percent on the worst paths after switching to BBR globally. Spotify reduced rebuffer ratio. Cloudflare uses BBR for selective edge endpoints. Netflix Open Connect uses BBR for long-haul fill traffic into ISP caches.

BBR caveats#

BBRv1 is unfair when sharing a bottleneck with CUBIC: the model-based pacing claims a larger share than a loss-based competitor will take. BBRv2 (Google, 2019) and BBRv3 (2023) add loss and ECN signals back in to improve coexistence, with retransmit rates an order of magnitude lower than v1.

Explicit Congestion Notification (ECN)#

ECN lets routers signal congestion before dropping packets. The IP header has two bits:

ECT(0) or ECT(1): the sender marks the packet ECN-capable.
CE: a router experiencing congestion sets this instead of dropping.

The receiver echoes a CE back via the TCP ECE flag; the sender treats it as a half-strength loss signal and reduces cwnd without anyone losing data.

DCTCP (Datacenter TCP) pushes ECN further: instead of halving on a single CE mark, it cuts cwnd by a fraction proportional to the recent fraction of CE-marked ACKs. This keeps datacenter queues tiny (a few KB) and yields predictable microsecond latencies.

L4S (Low Latency, Low Loss, Scalable throughput) is the 2023 IETF effort to bring DCTCP-style ECN to the public internet. It is in early Linux kernels and Apple's iOS 17 networking stack.

Linux: the `tcp_congestion_control` sysctl#

Modern Linux ships several algorithms compiled in. List and choose:

# What's available
sysctl net.ipv4.tcp_available_congestion_control
# net.ipv4.tcp_available_congestion_control = reno cubic bbr bbr2

# What's active (default)
sysctl net.ipv4.tcp_congestion_control
# net.ipv4.tcp_congestion_control = cubic

# Switch to BBR (kernel-wide)
sudo sysctl -w net.ipv4.tcp_congestion_control=bbr

# Persist across reboots
echo "net.ipv4.tcp_congestion_control=bbr" | sudo tee -a /etc/sysctl.conf

A specific socket can override the kernel default via setsockopt(TCP_CONGESTION, "bbr"), which is how envoy, ngnix, and the Cloudflare proxy choose per-connection. Pacing requires fq qdisc with BBR: tc qdisc add dev eth0 root fq.

Picking the right algorithm#

Workload	Best choice	Why
General Linux web server	CUBIC	Safe default, fair with other CUBIC flows
YouTube, video streaming over wifi/cell	BBR	Loss is non-congestion; BBR keeps pipe full
Datacenter intra-rack	DCTCP or BBR	Low queue depth, predictable RTT
HFT colocated trading	reno or cubic with low buffers	Predictable, well-studied tail
Lossy intercontinental link	BBR	CUBIC overreacts to drops
Mixed public internet with CUBIC neighbors	CUBIC or BBRv2/v3	BBRv1 starves CUBIC neighbors

How CDNs and streamers use this#

YouTube ran the original BBR rollout on youtube.com flows in 2016. The user-visible win was faster video startup and fewer rebuffers on cellular.
Netflix Open Connect boxes pull catalog updates over BBR to ISP edges; transcontinental fills run at near-line-rate where CUBIC would hover at half.
Cloudflare lets enterprise customers opt into BBR via dashboard; default remains CUBIC for compatibility with CUBIC-dominant public internet.
Akamai uses a custom variant called FAST-TCP on a subset of edges.

Common interview probes#

"Why does the first MB of a download take longer than the second MB?" Slow start. The window has not yet grown to fill the pipe.
"Why does my video stutter on wifi at the back of the cafe?" CUBIC reacts to non-congestion loss; signal-related drops halve the window unnecessarily. BBR would not.
"What is the bandwidth-delay product?" bandwidth * RTT. The amount of data needed in flight to keep the pipe full. The right cwnd target for any algorithm.
"What happens during TCP slow start in a TLS connection?" Slow start applies to every fresh TCP connection. Plus TLS adds 1 or 2 extra RTTs of handshake. The whole reason QUIC uses 0-RTT and TFO exists.

FAQ#

What is TCP congestion control?#

It is a sender-side algorithm that adjusts how fast a TCP sender pushes data into the network so flows share bandwidth fairly and routers do not collapse under queue overflow. It is distinct from flow control.

What is the difference between flow control and congestion control in TCP?#

Flow control protects the receiver via the advertised window. Congestion control protects the network via the sender-side congestion window adjusted from loss, RTT, or ECN signals.

How does TCP slow start work?#

After connection setup the congestion window starts small, typically 10 segments, then doubles every round-trip time until a loss event or a threshold is crossed. This exponential ramp probes capacity quickly.

Why did Linux switch to CUBIC and Google to BBR?#

CUBIC scales much better than Reno on high-bandwidth long-RTT links because its window grows on a cubic curve, not linearly. BBR models bandwidth and RTT directly and avoids reacting to non-congestion loss on wifi and cellular.

What is AIMD in TCP congestion control?#

AIMD is Additive Increase Multiplicative Decrease. After leaving slow start, the sender adds one segment to the window per RTT during congestion avoidance, and on a loss event it cuts the window roughly in half. This rule provably converges to fair sharing among competing flows.

Why is CUBIC the Linux default?#

CUBIC scales much better than Reno on high-bandwidth long-RTT paths. Its cubic growth curve rapidly returns close to the previous window after a drop and then probes more cautiously past it, so a transcontinental flow at 10 Gbps can recover full throughput in seconds instead of hours.

What problem does BBR solve over CUBIC?#

BBR ignores loss and models the path bandwidth and minimum RTT directly. On lossy wifi or cellular links where drops do not indicate congestion, CUBIC overreacts and stalls, while BBR keeps the pipe full. BBR also keeps router queues short, reducing tail latency.

What is the bandwidth delay product in TCP?#

BDP is the bottleneck bandwidth multiplied by the round-trip propagation time. It is the amount of data that fits in the pipe at one moment. BBR targets in-flight data equal to BDP so the link is fully utilized but the bottleneck queue stays empty.

What is the formula for TCP throughput?#

Throughput is roughly the minimum of the congestion window and the receive window divided by the round-trip time. Doubling the RTT halves the throughput, which is why CDNs and edge POPs matter so much.

How big is the initial congestion window?#

RFC 6928 raised it to 10 MSS, roughly 14 KB. Older kernels used 2 to 4. A higher initial window means small downloads finish in one RTT instead of needing a slow-start ramp.

When should I enable BBR?#

Enable BBR on servers that send large flows over lossy or long-RTT paths: video CDNs, software downloads, intercontinental replication. Stay on CUBIC if you share bottlenecks with many CUBIC clients and need fair coexistence.

What does ECN do?#

ECN lets routers signal congestion by marking a packet bit instead of dropping the packet. The sender treats the echoed mark like a soft loss and reduces the window, keeping latency low and avoiding actual retransmits.

TCP Deep Dive: the full TCP machinery congestion control sits inside
HTTP/3 and QUIC: QUIC ships its own congestion controller, BBR by default
Backpressure: the application-layer cousin of congestion control
Network Basics: the OSI / IP model TCP rides on

Quick reference#

A sender-side cheat sheet for TCP congestion control: the congestion window, the four classic phases, how Reno, CUBIC, and BBR differ, and the Linux knobs to flip.

Core variables#

Symbol	Meaning
`cwnd`	Sender's congestion window in bytes
`rwnd`	Receiver's advertised window
`ssthresh`	Slow-start threshold
`RTT`	Round-trip time
`RTprop`	Minimum (no-queue) RTT
`BtlBw`	Bottleneck bandwidth
`BDP`	`BtlBw * RTprop`, the bandwidth-delay product
`MSS`	Maximum segment size, typically 1460 bytes

A sender may have at most min(cwnd, rwnd) unacked bytes in flight.

Throughput formula#

throughput ~= min(cwnd, rwnd) / RTT

Double the RTT, halve the throughput. Why CDNs matter.

The four classic phases (Reno / NewReno)#

Phase	Rule	Trigger to leave
Slow start	`cwnd += MSS` per ACK (doubles per RTT)	`cwnd >= ssthresh` or loss
Congestion avoidance	`cwnd += MSS / cwnd` per ACK (+1 segment per RTT)	3 dupacks or RTO
Fast retransmit + fast recovery	retransmit, `ssthresh = cwnd/2`, `cwnd = ssthresh`	New ACK arrives
Timeout (RTO)	`ssthresh = cwnd/2`, `cwnd = 1 MSS`, slow start	New ACK arrives

AIMD is the rule that makes Reno fair: Additive Increase on every successful RTT, Multiplicative Decrease on every loss.

Algorithm cheat sheet#

Algo	Year	Style	Decrease	Default in
Tahoe	1988	Loss	`cwnd = 1`	Original
Reno	1990	Loss	`cwnd /= 2` (fast recovery)	Legacy BSD
NewReno	1999	Loss	as Reno, multi-loss aware	Older Linux
CUBIC	2008	Loss, cubic curve	`cwnd *= 0.7`	Linux 2.6.19+
Vegas	1994	RTT-based	proportional to RTT delta	Research
BIC	2005	Loss, binary search	adaptive	Linux 2.6.13-2.6.18
BBR (v1)	2016	Model-based	none (paces to BtlBw)	YouTube, Google
BBRv2 / v3	2019, 2023	Model + ECN + loss	gentle	Cloudflare, Google
DCTCP	2010	ECN-based	proportional to ECN fraction	Datacenter

Reno vs CUBIC vs BBR at a glance#

Property	Reno	CUBIC	BBR
Signal	Loss	Loss	Bandwidth + RTT
Growth shape	Linear	Cubic	Pacing at `BtlBw`
Behaves well on long fat pipes	No	Yes	Yes
Behaves well on lossy wifi/cell	No	No	Yes
Keeps queues short	No	No	Yes
Fair vs CUBIC neighbor	Yes	Yes	v1 unfair, v2/v3 fair
Where it ships	Legacy	Linux default	Google, YouTube, Cloudflare

Bandwidth-delay product#

BDP = BtlBw * RTprop

10 Gbps * 70 ms RTT = 87.5 MB, ~60000 MSS in flight to keep the pipe full. Reno takes ~30000 RTTs to recover that after one drop. CUBIC does it in seconds. BBR holds it as a target.

ECN basics#

Bit	Meaning
`ECT(0)` / `ECT(1)`	Sender marks packet ECN-capable
`CE`	Router stamps when congested instead of dropping
`ECE` (TCP flag)	Receiver echoes CE back to sender
`CWR` (TCP flag)	Sender acknowledges, has reduced cwnd

DCTCP uses the fraction of CE-marked ACKs to do proportional cuts. Sub-millisecond datacenter queues.

L4S generalizes DCTCP for the public internet using ECT(1) to signal scalable congestion control. Linux 5.x and iOS 17 have early support.

Linux knobs#

# List available algorithms
sysctl net.ipv4.tcp_available_congestion_control

# Current default
sysctl net.ipv4.tcp_congestion_control

# Switch to BBR
sudo sysctl -w net.ipv4.tcp_congestion_control=bbr

# BBR requires fq qdisc for pacing
sudo tc qdisc add dev eth0 root fq

# Other relevant sysctls
net.ipv4.tcp_slow_start_after_idle = 0   # don't reset cwnd after idle
net.ipv4.tcp_no_metrics_save     = 0     # cache RTT/cwnd between conns
net.ipv4.tcp_ecn                 = 1     # enable ECN
net.ipv4.tcp_window_scaling      = 1     # large windows for high BDP

Per-socket override:

setsockopt(fd, IPPROTO_TCP, TCP_CONGESTION, "bbr", 3);

Initial congestion window#

Spec	IW
RFC 2581 (1999)	2-4 MSS
RFC 3390 (2002)	up to 4 MSS
RFC 6928 (2013, Google)	10 MSS

The IW=10 change cut average page load time at Google by ~10 percent. Most modern stacks use 10.

Production examples#

Operator	Choice	Why
YouTube serving	BBR	lossy mobile, long RTT
Google Search	BBR	low tail latency
Cloudflare edge	CUBIC default, BBR opt-in	fairness across mixed traffic
Netflix Open Connect fills	BBR	bulk long-haul
Spotify backend	BBR	lossy mobile audio
AWS internal east-west	DCTCP-like	datacenter ECN

Common symptoms and causes#

Symptom	Likely cause
Slow first MB of a large file	Slow start ramp
Throughput collapses on wifi	CUBIC reacting to non-congestion loss
90th-percentile latency spikes	Bufferbloat at bottleneck
Inter-DC throughput tops out at ~Reno levels	Algorithm not switched to CUBIC/BBR
BBR sender starves a CUBIC sender	BBRv1 fairness; upgrade to BBRv2/v3

One-liner mental models#

Slow start: probe up fast.
AIMD: be polite on the way up, panic on the way down.
CUBIC: rush back to the last good window, then probe past it cautiously.
BBR: aim for BtlBw * RTprop, ignore the noise.
ECN: mark, do not drop.

Refs#

RFC 5681 - TCP Congestion Control
RFC 6928 - Increasing the TCP Initial Window
RFC 8312 - CUBIC for Fast and Long-Distance Networks
RFC 9438 - CUBIC update (2023)
Cardwell et al., 2016 - BBR: Congestion-Based Congestion Control
Alizadeh et al., 2010 - DCTCP
Jacobson, 1988 - Congestion Avoidance and Control

TCP Congestion Control#

The one-paragraph mental model#

Why TCP needs congestion control#

Reno: slow start, AIMD, fast recovery#

CUBIC: scaling to long fat pipes#

BBR: model-based, not loss-based#

ECN: a politer signal#

Where each one wins#

Why TCP needs congestion control#

The classic Reno state machine#

Pseudocode for Reno#

Why Reno is not enough: the long fat pipe problem#

CUBIC: cubic window growth#

BBR: model-based congestion control#

Why BBR matters#

BBR caveats#

Explicit Congestion Notification (ECN)#

Linux: the tcp_congestion_control sysctl#

Picking the right algorithm#

How CDNs and streamers use this#

Common interview probes#

FAQ#

What is TCP congestion control?#

What is the difference between flow control and congestion control in TCP?#

How does TCP slow start work?#

Why did Linux switch to CUBIC and Google to BBR?#

What is AIMD in TCP congestion control?#

Why is CUBIC the Linux default?#

What problem does BBR solve over CUBIC?#

What is the bandwidth delay product in TCP?#

What is the formula for TCP throughput?#

How big is the initial congestion window?#

When should I enable BBR?#

What does ECN do?#

Related Topics#

Quick reference#

Core variables#

Throughput formula#

The four classic phases (Reno / NewReno)#

Algorithm cheat sheet#

Reno vs CUBIC vs BBR at a glance#

Bandwidth-delay product#

ECN basics#

Linux knobs#

Initial congestion window#

Production examples#

Common symptoms and causes#

One-liner mental models#

Refs#

Linux: the `tcp_congestion_control` sysctl#