Facebook News Feed#
Problem statement (interviewer prompt)
Design the Facebook News Feed: aggregate posts from friends, pages, groups, and recommendations into a personalised feed. Optimise for p99 feed-open <300ms across 3B MAU with extreme variance in friend counts (a few to 5000+).
flowchart LR
U([User])
W[Write: post]
POST[(Posts)]
AGG([Aggregator])
RANK([Ranker])
TAO[(TAO graph cache)]
FEED[Feed API]
U -->|post| W --> POST
W --> TAO
U -->|read| FEED --> AGG --> TAO
AGG --> RANK --> U
classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;
class U client;
class W,FEED service;
class POST,TAO datastore;
class AGG,RANK compute;
flowchart TB
subgraph Clients
Web
Mobile
end
subgraph Edge
DNS
LB[L7 LB]
GW[GraphQL Gateway]
end
subgraph Write[Compose path]
COMP[Compose Service]
MEDIA[Media Service]
OBJ[(Haystack / S3)]
POSTS[(Posts store<br/>sharded MySQL)]
TAO[(TAO<br/>distributed graph cache)]
INGEST[[Kafka post events]]
end
subgraph Read[Read path - aggregator]
FEED([News Feed Aggregator])
LEAF[Leaf services<br/>per friend / per source]
RANK([Ranker - DL model])
SCORE[Scoring service]
HYD([Hydrator])
PAG[Cursor pagination]
end
subgraph Sources[Candidate sources]
FRIENDS[Friends posts]
PAGES[Pages followed]
GROUPS[Groups]
SUGG([Recommendations / video])
ADS[Ads]
end
subgraph Signals
AFF[Affinity scores]
DECAY[Time decay]
ENG[Past engagement features]
EMB([Embeddings store])
end
subgraph Safety
INTEG[Integrity / hate-speech filter]
DEDUP[Dedup]
FATIGUE[Source diversity / fatigue]
end
subgraph Realtime
NOTIF[Notification Service]
PUSH((APNS / FCM))
end
subgraph Storage
LIKES[(Likes / reactions)]
COMM[(Comments)]
ACT[(Activity log)]
end
Clients --> DNS --> LB --> GW
GW --> COMP --> POSTS
COMP --> MEDIA --> OBJ
COMP --> TAO
COMP --> INGEST --> NOTIF --> PUSH
GW --> FEED --> LEAF
LEAF --> Sources
Sources --> TAO
Sources --> POSTS
LEAF --> SCORE --> RANK
Signals --> RANK
RANK --> Safety --> HYD --> PAG --> Clients
LIKES --- HYD
COMM --- HYD
ACT --- LEAF
classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
classDef edge fill:#cffafe,stroke:#0e7490,stroke-width:1px,color:#0f172a;
classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
classDef datastore fill:#fee2e2,stroke:#991b1b,stroke-width:1px,color:#0f172a;
classDef cache fill:#fed7aa,stroke:#9a3412,stroke-width:1px,color:#0f172a;
classDef queue fill:#ede9fe,stroke:#5b21b6,stroke-width:1px,color:#0f172a;
classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
classDef storage fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#0f172a;
classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
classDef obs fill:#f3e8ff,stroke:#6b21a8,stroke-width:1px,color:#0f172a;
class LB,GW edge;
class COMP,MEDIA,LEAF,SCORE,PAG,FRIENDS,PAGES,GROUPS,ADS,AFF,DECAY,ENG,INTEG,DEDUP,FATIGUE,NOTIF service;
class POSTS,TAO,LIKES,COMM,ACT datastore;
class INGEST queue;
class FEED,RANK,HYD,SUGG,EMB compute;
class OBJ storage;
class PUSH external;
Aggregator pattern#
- Pull-on-read: at feed open, fan-out queries to leaves (one per source / friend bucket).
- Each leaf returns top-K candidates (recent + popular per source).
- Aggregator merges → candidate pool → ranker → top-N.
- Cache final feed page in user's recents for back-press / pagination.
TAO (The Associations and Objects)#
- Read-through write-through graph cache over MySQL.
- Objects (
type, id) and Associations (from, type, to) with counts. - Read-mostly; eventual consistency across regions.
Ranking#
- Thousands of features per candidate.
- Multi-task DNN: predicts P(like), P(comment), P(share), P(meaningful interaction).
- Combined into a value model with policy weights.
Caveats vs Twitter#
- Facebook leans pull (large per-friend variance, fewer celebs in friend graph).
- Twitter leans push for normal users (very long-tail celeb followers).
Glossary & fundamentals#
Concepts referenced in this design. Each row links to its canonical page; the tag column shows whether it is a high-level (HLD) or low-level (LLD) concept.
| Tag | Concept | What it is | Page |
|---|---|---|---|
HLD |
Load balancer / GSLB | L4/L7 traffic distribution and failover | load-balancer |
HLD |
Cache strategies | cache-aside, read/write-through, eviction | caching-strategies |
HLD |
Pub/Sub & message brokers | topics, consumer groups, delivery semantics | pub-sub-pattern |
HLD |
CAP / PACELC | C vs A under partition; L vs C otherwise | cap-pacelc |
LLD |
REST API design | verbs, statuses, pagination, errors | rest-api-design |
Quick reference#
Functional#
- Aggregate posts from friends, pages, groups, recommendations.
- Rank by relevance (Meaningful Social Interactions).
- Inject ads at known positions.
- React, comment, share, save, hide.
Non-functional#
- 3B+ MAU. Feed open p99 < 300 ms.
- Multi-region active-active for reads.
Capacity#
- Posts/day: ~5B (incl. comments, reacts).
- Reads/day: 100B+.
- Feed compute: thousands of features × millions of candidates evaluated per session.
Schema (conceptual)#
- TAO objects:
user,post,page,group,photo. - TAO assocs:
friends,likes,follows,posted. - Posts in sharded SQL by
owner_id. - Aggregations precomputed and cached.
Trade-offs#
- Pull-aggregator is more expensive per read but doesn't suffer the celeb fan-out problem.
- TAO sacrifices some consistency for cross-region read latency.
- Heavy ML demands GPU/TPU; staged ranking (cheap candidate gen → expensive scorer).
- Privacy & integrity complexity dwarfs the ranking complexity.
Refs#
- Facebook TAO paper (USENIX ATC '13).
- Haystack (small photo storage) paper.
- "How News Feed Works" engineering posts.
- ByteByteGo & Alex Xu Vol 2.
FAQ#
How does Facebook handle celebrity accounts in the feed?#
A hybrid model: regular accounts push posts to follower timelines, celebrities are pulled at read time. The feed merges pushed timeline with fresh celebrity posts on open.
What was EdgeRank and what replaced it?#
EdgeRank scored posts by affinity, weight, and time decay. It was replaced around 2013 by a learning-to-rank model that uses thousands of signals trained per user.
What is TAO in Facebook's architecture?#
TAO is Facebook's distributed graph cache layered over MySQL. It serves billions of social-graph reads per second with per-region replicas and write-through caching.
How does feed pagination work?#
Cursors point to a feed position, usually an opaque score plus tiebreaker. Newer items inserted above the cursor stay visible, and the API returns the next N candidates.
How do you hit p99 under 300 ms for feed open?#
Precompute personalized timelines, cache them in memory, parallelize fetches for candidates and ranking features, and short-circuit ranking on cache hit.