Build the core of a social network: users follow each other and post updates, and every user sees a fresh, fast home feed of posts from the people they follow - even when one of those people has 20 million followers.
Fan-out is delivering one thing to many destinations. Here, when a user makes one post, that single post has to reach the home feed of every one of their followers: one post from a user with 200 followers becomes 200 feed deliveries, and one post from a celebrity with 20 million followers becomes 20 million. There are two ways to pay that cost: do the delivery work when the post is written (fan-out-on-write) or do it when each follower opens their feed (fan-out-on-read). Choosing between them - and knowing when to use each - is the central decision of this design, and it is exactly the "hot user handling" we put in scope.
On CAP theorem: during a network partition a system can stay consistent (everyone sees exactly the same data) or stay available (everyone gets an answer), but not both. A newsfeed picks availability: if part of the system cannot talk to another part, we would rather show you a feed that is missing the last few seconds of posts than show you an error page. Nothing about a feed needs to be perfectly up to the millisecond. Eventual consistency is the price of that choice. When someone posts, their followers see it a few seconds later, not instantly, and for a brief moment two followers might see slightly different feeds. Once the system catches up, everyone converges on the same posts. That is completely fine for social content - and it would be completely wrong for, say, a bank balance, which is why other designs make the opposite call. Read-your-own-writes is the one consistency promise we do keep: after you hit Post, you see your own post at once. Waiting even two seconds to see your own words feels broken, so the author's reads are served from the freshest copy while everyone else catches up within the sync window.
The numbers have a surprising shape: the system looks read-heavy at the API (far more feed loads than posts) but is write-heavy at the storage layer, because every post multiplies into hundreds of feed-entry writes. A good design has to serve both faces at once. For the write side, the load is a firehose of tiny, independent appends - "add this post id to this user's feed" - each touching exactly one user's data, with no transactions spanning rows and no joins. That is the textbook case for a horizontally scalable NoSQL store (DynamoDB or Cassandra) partitioned by user_id: every append lands on one partition, and adding servers adds throughput. Compare this with Design Calendly and Design Dropbox, which chose PostgreSQL because they needed relational guarantees - Calendly must make two overlapping bookings impossible, which takes transactions and constraints. A feed entry needs no such protection; if one is written twice or a few seconds late, nothing breaks. So we trade the relational guarantees we do not need for the raw append throughput we very much do. For the read side, the trick is that the Feed Store IS the answer to the feed query, computed ahead of time. A feed load does not compute anything interesting - it reads the precomputed list, tops it up from a couple of caches, and returns. Redis absorbs the first-page loads of active users, and the recent posts of hot accounts are cached once and served to the millions of readers who follow them. Two more choices fall straight out of the estimates. First, the bytes-versus-metadata split, same as Design Dropbox: images and video live in blob storage behind a CDN, and the database stores only URLs - that is why 20M posts/day is a mere ~20GB/day of database growth. Second, denormalized counters: we store like_count on the post row rather than counting reaction rows on every render, paying a little extra work on each write to keep the massively more frequent reads cheap.
WRITE PATH - making a post
[Client] --(media via presigned URLs)--> [Blob Storage] --> [CDN]
[Client] --> [API Gateway] --> [Post Service] --> [Posts DB]
|
v (ack the poster immediately)
[Fan-out Queue]
|
v
[Fan-out Workers] --who follows X?--> [Follow Graph]
|
v append (post_id, author, ts), batched
[Feed Store (NoSQL, per-user partitions)]
(hot accounts with huge followings are NOT fanned out - see read path)
READ PATH - opening the app
[Client] --> [API Gateway] --> [Feed Service] --> [Feed Cache (Redis)]
| (miss)
+--> [Feed Store] newest ~50 pointer entries
+--> [Posts DB] recent posts of followed
| hot accounts (heavily cached)
v
merge by time -> hydrate ids into posts -> return page
(client fetches media bytes from the CDN separately)Queue: a durable buffer between "the post was saved" and "the post reached every follower's feed." The poster gets their acknowledgment the moment the post row is written; the 200 (or 20 million) feed deliveries happen afterwards, in the background. The queue absorbs bursts (a big event where everyone posts at once), lets us retry failed deliveries safely, and means a slow fan-out never makes posting feel slow. Hydration: feed entries are just pointers - a post_id, an author_id, and a timestamp. Turning that list of ids into actual posts (text, media URLs, like counts) is called hydrating, and it is one cheap batched lookup against the Posts DB. Keeping feeds as pointers and hydrating at read time is what makes editing and deleting posts trivial: there is only ever one real copy of the post.
When a post is fanned out to 200 followers, we could copy the whole post (text, media URLs, counts) into all 200 feeds, or we could write 200 tiny pointers that all reference the one real post. Pointers win for three reasons. Size: a full post row is ~1KB; a pointer is ~100 bytes. At 4 billion feed deliveries a day, copies would be ~4TB/day of feed writes versus ~400GB/day for pointers - a 10x difference on the most write-amplified table in the system. Change in one place: if the author edits or deletes the post, or the like count changes, there is exactly one row to update. With copies, a delete would mean hunting down 200 - or for a celebrity, 20 million - embedded duplicates. With pointers, hydration simply skips a post marked deleted, and every feed is instantly correct. Capping: because entries are tiny and reference-only, we can keep each user's feed trimmed to the newest ~500 entries without losing anything - older posts still exist in the Posts DB and remain reachable from profile pages. The Feed Store stays a small, fast, fixed-size structure per user instead of growing forever.
Database Schemas (NoSQL: DynamoDB / Cassandra - shown as key design)
- Posts (partitioned by author)
PK: author_id, SK: ts (newest first)
Attributes: post_id, text, media_urls[], like_count, comment_count, deleted
GSI: post_id -> item (direct lookup, used by hydration and permalinks)
-- one row per post; media bytes live in blob storage, only URLs here
-- deleted is a tombstone: hydration skips it, no feed cleanup needed
- Follow Graph
Following table -> PK: user_id, SK: followee_id ("who do I follow")
Reverse index -> PK: followee_id, SK: follower_id ("who follows X")
Attributes: created_at, followee_is_hot
-- fan-out cannot exist without the reverse index: when X posts, the
-- workers need X's followers, which the forward table cannot answer
- Feed Store (precomputed home feeds - pointers only)
PK: user_id, SK: ts#post_id (newest first)
Attributes: author_id
-- capped at the newest ~500 entries per user; older entries age out
-- ~100 bytes per entry because it holds ids, never post content
-- rebuildable from Posts + Follow Graph; losing it loses no posts
- Reactions
PK: post_id, SK: user_id
Attributes: reaction_type ('like' | 'love' | ...), ts
GSI: user_id -> posts this user reacted to (activity history)
-- one row per (post, user): a like is idempotent by construction
- Comments (flat - nesting is out of scope)
PK: post_id, SK: ts#comment_id
Attributes: user_id, text
Redis keys
- Feed Cache (first hydrated page per active user)
Key: "feed:{user_id}" -> [ {post...}, {post...} ]
TTL: ~60s; also invalidated when fan-out appends for this user
- Hot-account recent posts (the pull path, shared by millions of readers)
Key: "recent:{author_id}" -> [post_id, post_id, ...]
TTL: ~30s
- Buffered reaction counters (viral-post hot key protection)
Key: "likes:{post_id}" -> pending increments
Flushed to the Posts DB in batches every few seconds{
"text": "Sunset from the pier",
"media_urls": [
"https://cdn.example.com/media/m_7f2a91.jpg"
]
}{
"post_id": "p_8f3e2d1c",
"author_id": "u_123",
"ts": "2026-07-05T18:04:11Z",
"visibility": "followers"
}
// The media was already uploaded directly to blob storage via presigned
// URLs (see Design Dropbox for that flow) - this call stores only text and
// pointers. An Idempotency-Key header means a retried request never
// double-posts. Fan-out to followers happens asynchronously after this
// response returns.{
"items": [
{
"post_id": "p_9a1b44",
"author": { "user_id": "u_456", "name": "dana" },
"ts": "2026-07-05T18:03:40Z",
"text": "shipping day!",
"media_urls": [],
"like_count": 41,
"viewer_has_liked": false
},
{
"post_id": "p_77c2d0",
"author": { "user_id": "u_bigband", "name": "The Big Band" },
"ts": "2026-07-05T17:58:02Z",
"text": "Tour dates are live",
"media_urls": ["https://cdn.example.com/media/m_1182.jpg"],
"like_count": 182304,
"viewer_has_liked": true
}
],
"next_cursor": "2026-07-05T17:58:02Z#p_77c2d0",
"has_more": true
}
// The first item came from this user's precomputed Feed Store entries.
// The second came from the pull path (a hot account, never fanned out)
// and was merged in by timestamp. Its like_count is approximate - it is
// served from the buffered counter.{
"type": "like"
}{
"ok": true,
"like_count": 182305
}
// Idempotent: liking a post you already liked is a no-op and returns the
// same result. On viral posts the returned count is approximate (buffered
// in Redis, flushed in batches, reconciled periodically).{
"following": true,
"backfilled_posts": 10
}
// On follow, the followee's ~10 most recent post ids are copied into your
// feed so they appear immediately - otherwise you would see nothing from
// them until their next post.// 429 Too Many Requests - write endpoints are rate limited per user
{
"error": "rate_limited",
"message": "Too many posts. Try again in a minute.",
"retry_after_seconds": 60
}
// 404 Not Found - post does not exist OR is not visible to the caller
{
"error": "not_found",
"message": "Post not found"
}
// Follower-only posts return the same 404 as nonexistent ones on purpose,
// so outsiders cannot probe which post ids exist.Fan-out-on-write: deliver the post to every follower's feed at the moment it is posted. The write is expensive (one append per follower) but every later feed read is a cheap, precomputed lookup. This is the right default because feeds are read far more often than posts are written. Fan-out-on-read: deliver nothing up front; when a follower opens their feed, go fetch the recent posts of the people they follow. The write is free but every read does real work. We reserve this for hot accounts, where fan-out-on-write would mean millions of writes per post - and we lean on caching, since millions of readers want the same few posts. Cursor: a bookmark marking exactly where the last page ended - here, the timestamp plus post id of the last item. The next page asks for "everything older than this." Unlike page numbers, a cursor stays correct while new posts pile on top of the feed. The same idea drives sync in Design Dropbox; here it drives scrolling.