Pub/Sub Pattern#

Problem statement (interviewer prompt)

Design a messaging substrate that decouples producers from consumers. It should support topic-based fan-out, per-key ordering, at-least-once delivery (with optional exactly-once), bounded retention, and consumer groups for parallel processing.

flowchart LR
  P1[Publisher A]
  P2[Publisher B]
  B((Broker / Topic))
  S1[Subscriber 1]
  S2[Subscriber 2]
  S3[Subscriber 3]
  P1 --> B
  P2 --> B
  B --> S1
  B --> S2
  B --> S3

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;
    class P1,P2,S1,S2,S3 service;
    class B queue;

Publishers don't know subscribers; the broker fan-outs messages to all (or to a chosen group).

flowchart TB
  subgraph Producers
    P1[Service A]
    P2[Service B]
    P3([CDC / Debezium])
  end

  subgraph Broker[Broker Cluster]
    direction TB
    subgraph Topic_orders[Topic: orders]
      P0[Partition 0]
      P1p[Partition 1]
      P2p[Partition 2]
      P3p[Partition 3]
    end
    META[[Controller / Metadata<br/>Kafka KRaft, Zookeeper]]
    REPL[Replication ISR<br/>RF=3]
  end

  subgraph Storage
    LOG[Append-only log /<br/>segment files]
    IDX[Index per partition]
    TIER[Tiered storage S3<br/>cold segments]
  end

  subgraph Consumers
    subgraph CG_billing[Consumer Group: billing]
      CB1[c1 partitions 0,1]
      CB2[c2 partitions 2,3]
    end
    subgraph CG_analytics[Consumer Group: analytics]
      CA1[c1 partitions 0-3]
    end
    DLQ[(Dead Letter Topic)]
  end

  subgraph Semantics
    AL[At-Least-Once]
    AM[At-Most-Once]
    EO[Exactly-Once<br/>idempotent + tx]
    ORD[Per-partition Order]
    RT[Retention by time/size]
  end

  P1 --> Topic_orders
  P2 --> Topic_orders
  P3 --> Topic_orders
  Topic_orders --> LOG
  LOG --> IDX
  LOG -. age out .-> TIER
  META -. assign leaders .-> Topic_orders
  REPL --- Topic_orders
  Topic_orders --> CG_billing
  Topic_orders --> CG_analytics
  CG_billing -. poison msg .-> DLQ

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
    classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
    classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;
    class P1,P2,P0,P1p,P2p,P3p,REPL,LOG,IDX,CB1,CB2,CA1,AL,AM,EO,ORD,RT service;
    class DLQ datastore;
    class META queue;
    class P3 compute;
    class TIER storage;

Routing models#

Topic (Kafka, NATS, Pub/Sub): publisher writes once, multiple consumer groups read independently.
Queue / Work (SQS, RabbitMQ direct): each message goes to exactly one consumer of a group.
Topic + filters (NATS subjects, SNS attributes): selective subscribe.

Delivery semantics#

At-most-once: fire and forget.
At-least-once: ack after processing; duplicates possible - make consumers idempotent.
Exactly-once: transactional producer + idempotent consumer + read-process-write tx (Kafka EOS).

Ordering#

Per-partition / per-key only. To preserve order, use stable partition key (e.g., user_id).

Backpressure & DLQ#

Pull (Kafka) vs push (RabbitMQ); pull naturally backpressures.
Retries with exponential backoff; after N failures → DLQ.

Glossary & fundamentals#

Concepts referenced in this design. Each row links to its canonical page; the tag column shows whether it is a high-level (HLD) or low-level (LLD) concept.

Tag	Concept	What it is	Page
`HLD`	Pub/Sub & message brokers	topics, consumer groups, delivery semantics	pub-sub-pattern
`HLD`	Leader/follower replication	sync/semi-sync/async replication, failover	replication-leader-follower
`HLD`	Idempotency & retries	safe re-execution, backoff + jitter	idempotency-retries
`HLD`	Resilience patterns	timeout, retry, breaker, bulkhead, backpressure	resilience-patterns
`HLD`	Change Data Capture	WAL/binlog tailing, outbox publishing	change-data-capture

Quick reference#

Why use it#

Decouple producers from consumers.
Buffer bursts; consumers process at their own pace.
Fan-out: one event drives many side-effects (search index, cache invalidation, analytics).

Broker comparison (rough)#

	Kafka	RabbitMQ	SQS	Pub/Sub (GCP)	NATS JetStream
Model	log, partitioned	exchange + queue	queue	log-like	log
Order	per-partition	per-queue	best effort (FIFO opt)	per-key	per-subject
Retention	days-months	until consumed	4 days max	7 days	configurable
Throughput	very high	medium	high	very high	very high
Use case	event streaming	task queue	task queue	event streaming	edge messaging

Patterns#

Outbox pattern: write DB row + outbox event in same tx; relay to broker.
Saga: long-running tx via events + compensations.
Event sourcing: state = replay of events.

Pitfalls#

Schema evolution: use Avro/Protobuf + schema registry, enforce compatibility.
Hot partitions: bad key choice → uneven load.
Consumer lag: alert on it.

Refs#

Kafka design doc (LinkedIn), Confluent EOS post, RabbitMQ in Depth (book), AWS SQS/SNS architecture, NATS JetStream docs.

FAQ#

What is the pub/sub pattern?#

Publishers send messages to a broker topic without knowing the subscribers. Subscribers receive every message they care about, decoupling producers from consumers.

Pub/sub vs message queue?#

A queue delivers each message to exactly one consumer. Pub/sub fans the same message out to many subscribers, each on its own offset, ideal for event-driven systems.

What is a consumer group?#

A consumer group splits a topic's partitions across its members. Each partition goes to one member at a time, giving parallel processing with per-partition ordering.

How do you get exactly-once delivery?#

Combine an idempotent consumer (inbox table or dedup key) with at-least-once delivery. Brokers like Kafka also offer transactional writes for end-to-end exactly-once.

Which pub/sub system should I pick?#

Pick Kafka or Pulsar for durable, partitioned logs at scale. Pick SNS, Google Pub/Sub, or NATS for simpler hosted fan-out. Pick Redis Streams for low-latency, small footprints.

Event Sourcing and CQRS: event sourcing relies on pub/sub as the transport for distributing domain events to subscribers
Batch and Stream Processing: pub/sub topics feed real-time stream processing pipelines for immediate event handling
Change Data Capture: CDC pipelines publish database change events to pub/sub topics for downstream consumers