How does a chat system deliver a message to a user who is offline?

By storing before it notifies. When a message is sent, the server first writes a durable copy into each recipient’s inbox — a per-recipient queue — and only then attempts to push it over a live WebSocket. If the recipient is offline there is no socket to push to, but the copy is already safely stored. When their device reconnects, it sends its cursor (cur_max_message_id, the id of the newest message it has seen) and the server replays every stored message with a larger id — exactly the gap it missed, no duplicates. This is why the order of operations matters: a naive push-only system treats the socket as the delivery guarantee, so anything sent while the socket is down is lost. Store-then-notify treats the socket as a fast path and the durable inbox plus cursor as the real guarantee, so being offline is a delay, never a loss.

WebSocket vs long polling vs short polling for chat — which and why?

A chat system needs the server to push the instant a message arrives, and only a WebSocket does that well. Short polling has the client ask on a timer, so you are stuck choosing between latency (poll slowly, messages lag) and load (poll fast, and millions of idle clients hammer the server with mostly-empty requests). Long polling improves latency by having the server hold each request open until it has something, but it is awkward for the server to push repeatedly and it churns through reconnects. A WebSocket upgrades a single connection once and is then full-duplex — either side can send at any time — giving true server push over one cheap long-lived socket at the lowest latency. Its cost is that connections become stateful: the fleet is sized by how many are open at once, not by request rate. Long polling remains the standard fallback when a proxy blocks WebSockets.

Why do chat systems use a key-value store like Cassandra instead of a relational database?

Because the workload is a firehose of simple writes and recent-heavy reads that grows without end, which is exactly what wide-column key-value stores (Cassandra, ScyllaDB, HBase) are built for and where a single relational database struggles. Discord, explaining their move, put it bluntly: “Cassandra was the only database that fulfilled all of our requirements. We can just add nodes to scale it and it can tolerate a loss of nodes without any impact on the application.” The data is modeled as a partition key plus a sort key, and to keep any one conversation’s partition from growing unbounded it is bucketed by time — partitioned on (channel, time-bucket) — so each partition holds a bounded window and reads come back already sorted by message id. It also matches the access pattern: history reads are long-tailed, “usually very recent only,” so the newest bucket stays hot in cache while years of older buckets sit cheaply cold on disk.

How do you keep chat messages in the same order on every device?

By ordering on an id that is itself a sequence, not on a wall-clock timestamp. Sorting by created_at fails because two different chat servers can stamp the same millisecond, and their clocks drift apart by milliseconds, so a message that truly came second can carry an earlier timestamp — and different devices then disagree about the order. The fix is a Snowflake id: pack a timestamp into the high bits, a machine id into the middle, and a per-machine counter into the low bits, so ids are roughly time-ordered and guaranteed unique, minted on each server with no central coordinator. Discord uses exactly this — “every ID on Discord is actually a Snowflake (chronologically sortable).” Sort by message id and every device agrees on the sequence, permanently.

How does presence (online status) work at scale?

Presence is tracked by heartbeat and fanned out over pub/sub. A client with an open socket pings every few seconds; the server marks them online as long as the pings keep coming, keeping this in memory because it is soft, fast-changing state. The subtlety is flap suppression: mobile sockets drop constantly for a second without the person going anywhere, so the server waits a window — a few missed beats, say 30 seconds — before flipping anyone offline, absorbing brief flaps and only reacting to sustained silence. When a status does change, broadcasting it to every viewer with a query would melt under load, so changes are published to a pub/sub channel that every interested server subscribes to. Slack reported that moving presence to pub/sub cut the presence events clients received by a factor of five. Only ephemeral signals (presence, typing) ride this best-effort bus — real messages always take the durable inbox.

Design a Chat System

Two people, a message, and the expectation that it arrives instantly, in order, and never vanishes — even when the recipient is in a tunnel. Built from one polling request up to a stateful WebSocket fleet over a durable inbox: the commit-first envelope that sizes everything by open connections (not requests), the honest transport fork (short poll vs long poll vs WebSocket), a session registry that finds a user across the fleet, store-then-notify with per-recipient inbox queues, Snowflake IDs because created_at collides, a bucketed key-value store for history because reads are long-tailed, the signature offline-delivery exhibit where a per-device cursor replays exactly the gap, presence via heartbeat + flap-suppression + pub/sub fanout, and a failure sweep whose surprising box is a chat server dying with all its connections.

System design · Systems. The source ↗

A free, interactive, animated visual explainer of Design a Chat System — built to be understood, not skimmed.

Questions

How does a chat system deliver a message to a user who is offline?: By storing before it notifies. When a message is sent, the server first writes a durable copy into each recipient’s inbox — a per-recipient queue — and only then attempts to push it over a live WebSocket. If the recipient is offline there is no socket to push to, but the copy is already safely stored. When their device reconnects, it sends its cursor (cur_max_message_id, the id of the newest message it has seen) and the server replays every stored message with a larger id — exactly the gap it missed, no duplicates. This is why the order of operations matters: a naive push-only system treats the socket as the delivery guarantee, so anything sent while the socket is down is lost. Store-then-notify treats the socket as a fast path and the durable inbox plus cursor as the real guarantee, so being offline is a delay, never a loss.
WebSocket vs long polling vs short polling for chat — which and why?: A chat system needs the server to push the instant a message arrives, and only a WebSocket does that well. Short polling has the client ask on a timer, so you are stuck choosing between latency (poll slowly, messages lag) and load (poll fast, and millions of idle clients hammer the server with mostly-empty requests). Long polling improves latency by having the server hold each request open until it has something, but it is awkward for the server to push repeatedly and it churns through reconnects. A WebSocket upgrades a single connection once and is then full-duplex — either side can send at any time — giving true server push over one cheap long-lived socket at the lowest latency. Its cost is that connections become stateful: the fleet is sized by how many are open at once, not by request rate. Long polling remains the standard fallback when a proxy blocks WebSockets.
Why do chat systems use a key-value store like Cassandra instead of a relational database?: Because the workload is a firehose of simple writes and recent-heavy reads that grows without end, which is exactly what wide-column key-value stores (Cassandra, ScyllaDB, HBase) are built for and where a single relational database struggles. Discord, explaining their move, put it bluntly: “Cassandra was the only database that fulfilled all of our requirements. We can just add nodes to scale it and it can tolerate a loss of nodes without any impact on the application.” The data is modeled as a partition key plus a sort key, and to keep any one conversation’s partition from growing unbounded it is bucketed by time — partitioned on (channel, time-bucket) — so each partition holds a bounded window and reads come back already sorted by message id. It also matches the access pattern: history reads are long-tailed, “usually very recent only,” so the newest bucket stays hot in cache while years of older buckets sit cheaply cold on disk.
How do you keep chat messages in the same order on every device?: By ordering on an id that is itself a sequence, not on a wall-clock timestamp. Sorting by created_at fails because two different chat servers can stamp the same millisecond, and their clocks drift apart by milliseconds, so a message that truly came second can carry an earlier timestamp — and different devices then disagree about the order. The fix is a Snowflake id: pack a timestamp into the high bits, a machine id into the middle, and a per-machine counter into the low bits, so ids are roughly time-ordered and guaranteed unique, minted on each server with no central coordinator. Discord uses exactly this — “every ID on Discord is actually a Snowflake (chronologically sortable).” Sort by message id and every device agrees on the sequence, permanently.
How does presence (online status) work at scale?: Presence is tracked by heartbeat and fanned out over pub/sub. A client with an open socket pings every few seconds; the server marks them online as long as the pings keep coming, keeping this in memory because it is soft, fast-changing state. The subtlety is flap suppression: mobile sockets drop constantly for a second without the person going anywhere, so the server waits a window — a few missed beats, say 30 seconds — before flipping anyone offline, absorbing brief flaps and only reacting to sustained silence. When a status does change, broadcasting it to every viewer with a query would melt under load, so changes are published to a pub/sub channel that every interested server subscribes to. Slack reported that moving presence to pub/sub cut the presence events clients received by a factor of five. Only ephemeral signals (presence, typing) ride this best-effort bus — real messages always take the durable inbox.

Design a Chat System

Questions

Related explainers