Why does an auto-increment ID fail in a distributed system?

A single auto-increment column relies on one machine owning the counter. Shard the database and there is no shared counter, so every shard independently mints 1, 2, 3 and IDs collide. Routing all writes through one coordinator to keep the counter global just re-creates the bottleneck and single point of failure you sharded to escape, and two coordinators risk a split-brain that hands the same number out twice. Sequential IDs also leak your row count and are trivially enumerable in URLs.

What is a Snowflake ID and how are the 64 bits laid out?

Snowflake is Twitter’s 64-bit unique ID: 1 unused sign bit, then 41 bits of millisecond timestamp (about 69 years from a custom epoch), 10 bits of machine id (1024 workers, often split 5 datacenter + 5 worker), and 12 bits of per-millisecond sequence (4096 IDs per machine per millisecond). Time in the high bits makes IDs sort by age as plain integers; the distinct machine-id slot makes two machines unable to collide, so no coordination is needed.

What is the difference between UUID v4 and UUID v7?

UUIDv4 is 122 random bits (of 128): unique and coordination-free, but unsortable and 128 bits, which RFC 9562 says gives "poor database-index locality" for write-heavy tables. UUIDv7 keeps local generation but moves a 48-bit Unix-millisecond timestamp into the high bits so values are time-ordered and index-dense; the RFC recommends implementations "utilize UUIDv7 instead of UUIDv1 and UUIDv6 if possible."

How does clock skew break Snowflake, and how is it handled?

Snowflake assumes time only moves forward. When NTP steps the clock backward, the worker re-lives milliseconds it already stamped IDs for; since the sequence resets each millisecond, it can mint a byte-for-byte duplicate of an earlier ID. The fix is to remember the last timestamp issued and never issue at one less than or equal to it — either refuse to generate IDs until the clock catches up (Twitter’s choice) or wait/sleep until it does. Both trade a few milliseconds of availability for correctness; running NTP in slew-only mode avoids the skew entirely.

How many IDs per second can a Snowflake generator produce?

The 12-bit sequence gives 4096 IDs per machine per millisecond, i.e. about 4.1 million per second per worker before it must wait for the next millisecond. With 1024 machine slots the whole fleet can mint over 4 billion per second collision-free. Widening or narrowing the sequence field trades this per-worker ceiling against timestamp range and machine count — the three fields must sum to 63 bits.

Unique IDs at Scale

A single auto-increment counter is trivial — until you shard and it becomes the hardest thing in the system. Watch the fix: pack a clock, a machine id, and a per-millisecond counter into 64 bits, with a scrubbable Snowflake bit-strip and clock-skew you can break.

Concept · Systems. The source ↗

A free, interactive, animated visual explainer of Unique IDs at Scale — built to be understood, not skimmed.

Questions

Why does an auto-increment ID fail in a distributed system?: A single auto-increment column relies on one machine owning the counter. Shard the database and there is no shared counter, so every shard independently mints 1, 2, 3 and IDs collide. Routing all writes through one coordinator to keep the counter global just re-creates the bottleneck and single point of failure you sharded to escape, and two coordinators risk a split-brain that hands the same number out twice. Sequential IDs also leak your row count and are trivially enumerable in URLs.
What is a Snowflake ID and how are the 64 bits laid out?: Snowflake is Twitter’s 64-bit unique ID: 1 unused sign bit, then 41 bits of millisecond timestamp (about 69 years from a custom epoch), 10 bits of machine id (1024 workers, often split 5 datacenter + 5 worker), and 12 bits of per-millisecond sequence (4096 IDs per machine per millisecond). Time in the high bits makes IDs sort by age as plain integers; the distinct machine-id slot makes two machines unable to collide, so no coordination is needed.
What is the difference between UUID v4 and UUID v7?: UUIDv4 is 122 random bits (of 128): unique and coordination-free, but unsortable and 128 bits, which RFC 9562 says gives "poor database-index locality" for write-heavy tables. UUIDv7 keeps local generation but moves a 48-bit Unix-millisecond timestamp into the high bits so values are time-ordered and index-dense; the RFC recommends implementations "utilize UUIDv7 instead of UUIDv1 and UUIDv6 if possible."
How does clock skew break Snowflake, and how is it handled?: Snowflake assumes time only moves forward. When NTP steps the clock backward, the worker re-lives milliseconds it already stamped IDs for; since the sequence resets each millisecond, it can mint a byte-for-byte duplicate of an earlier ID. The fix is to remember the last timestamp issued and never issue at one less than or equal to it — either refuse to generate IDs until the clock catches up (Twitter’s choice) or wait/sleep until it does. Both trade a few milliseconds of availability for correctness; running NTP in slew-only mode avoids the skew entirely.
How many IDs per second can a Snowflake generator produce?: The 12-bit sequence gives 4096 IDs per machine per millisecond, i.e. about 4.1 million per second per worker before it must wait for the next millisecond. With 1024 machine slots the whole fleet can mint over 4 billion per second collision-free. Widening or narrowing the sequence field trades this per-worker ceiling against timestamp range and machine count — the three fields must sum to 63 bits.

Unique IDs at Scale

Questions

Related explainers