How do you scale a web app from one server to millions of users?

Not in one leap but as an ordered sequence, where each stage is forced by the bottleneck the last one exposed. Start with everything on one box; give the database its own machine; put many stateless web servers behind a load balancer; add read replicas; cache the hot reads and push static assets to a CDN; autoscale the fleet across availability zones; shard the database once one primary can no longer take the writes; and decouple write bursts behind a queue. Each move buys roughly an order of magnitude and hands you the next bottleneck.

What is the difference between vertical and horizontal scaling?

Vertical scaling ("scale up") means buying a bigger machine — more CPU, RAM, and disk on one box. It is the simplest fix and needs no code changes, but it has a hard ceiling (the biggest instance you can rent) and a single point of failure. Horizontal scaling ("scale out") means adding more machines and spreading load across them. It scales far past any single box and removes the single point of failure, but it forces real complexity: a load balancer, stateless servers, and a story for shared data.

Why do read replicas cause stale reads (replication lag)?

A read replica copies the primary’s changes asynchronously — the primary acknowledges a write and streams it to replicas a moment later. During that window a read served by a replica can return data older than the write that just committed, so a user who updates their profile and immediately reloads may see the old value. The honest fixes are to route reads that must be current back to the primary (read-your-writes), or to accept and design around the lag where staleness is harmless.

When should you shard a database?

Sharding is the last resort, taken only when a single primary can no longer hold the write throughput or dataset — replicas scale reads, but every write still lands on one primary. Sharding splits the data across independent primaries by a shard key, so writes spread too. The cost is steep: cross-shard queries and joins get hard, transactions across shards are painful, a hot key can overload one shard, and re-sharding a live system is one of the hardest operations in the field. Exhaust caching, replicas, and queues first.

Why must the web tier be stateless before you can load balance?

A load balancer spreads each request across whatever server is free, so consecutive requests from one user can land on different servers. If session state (who is logged in, a cart) lives in a server’s local memory, a request routed to a different server has no session — the user appears randomly logged out. Making the tier stateless means moving that session state to a shared store (a cache like Redis, or the database) that every server reads, so any server can handle any request. Externalizing sessions is the step that unlocks horizontal scaling of the web tier.

From One Server to Millions of Users

The scaling journey as a playable timeline. Start with everything on one box, then let each real bottleneck force the next move — split the database, load-balance a stateless web tier, add read replicas, cache, autoscale across zones, shard, and decouple write bursts with queues.

Concept · Systems. The source ↗

A free, interactive, animated visual explainer of From One Server to Millions of Users — built to be understood, not skimmed.

Questions

How do you scale a web app from one server to millions of users?: Not in one leap but as an ordered sequence, where each stage is forced by the bottleneck the last one exposed. Start with everything on one box; give the database its own machine; put many stateless web servers behind a load balancer; add read replicas; cache the hot reads and push static assets to a CDN; autoscale the fleet across availability zones; shard the database once one primary can no longer take the writes; and decouple write bursts behind a queue. Each move buys roughly an order of magnitude and hands you the next bottleneck.
What is the difference between vertical and horizontal scaling?: Vertical scaling ("scale up") means buying a bigger machine — more CPU, RAM, and disk on one box. It is the simplest fix and needs no code changes, but it has a hard ceiling (the biggest instance you can rent) and a single point of failure. Horizontal scaling ("scale out") means adding more machines and spreading load across them. It scales far past any single box and removes the single point of failure, but it forces real complexity: a load balancer, stateless servers, and a story for shared data.
Why do read replicas cause stale reads (replication lag)?: A read replica copies the primary’s changes asynchronously — the primary acknowledges a write and streams it to replicas a moment later. During that window a read served by a replica can return data older than the write that just committed, so a user who updates their profile and immediately reloads may see the old value. The honest fixes are to route reads that must be current back to the primary (read-your-writes), or to accept and design around the lag where staleness is harmless.
When should you shard a database?: Sharding is the last resort, taken only when a single primary can no longer hold the write throughput or dataset — replicas scale reads, but every write still lands on one primary. Sharding splits the data across independent primaries by a shard key, so writes spread too. The cost is steep: cross-shard queries and joins get hard, transactions across shards are painful, a hot key can overload one shard, and re-sharding a live system is one of the hardest operations in the field. Exhaust caching, replicas, and queues first.
Why must the web tier be stateless before you can load balance?: A load balancer spreads each request across whatever server is free, so consecutive requests from one user can land on different servers. If session state (who is logged in, a cart) lives in a server’s local memory, a request routed to a different server has no session — the user appears randomly logged out. Making the tier stateless means moving that session state to a shared store (a cache like Redis, or the database) that every server reads, so any server can handle any request. Externalizing sessions is the step that unlocks horizontal scaling of the web tier.

From One Server to Millions of Users

Questions

Related explainers