Storage

Keeping bytes safe and findable at scale: durability engines, erasure coding, replication, and the metadata that locates every object.

Explainers

• Design a Transaction Log on Object Storage — A cloud bucket can swap one file atomically and never many at once — so how does a pile of Parquet files on S3 become an ACID table? The Delta-shaped question, built up from first principles: the naive folder and the three ways it dies, the write-ahead log as the source of truth, the put-if-absent commit (and why S3 needs a tiny coordinator), optimistic concurrency with the conflict rules raced live, checkpoints against a computed cold-read cost, compaction, time travel, and the log doubling as a message queue — then an honest Delta vs Iceberg vs Hudi.
• A Write-Ahead Log You Can Implement in an Hour — The low-level-design round asks you to make a key-value store survive a crash — and the whole answer is one small file. Append a length + CRC32 framed record before you touch memory, fsync on the durability dial you choose, and on restart replay the log and drop the torn tail. We author the ~120-line Python module, crash it in a child process, and walk it top to bottom.
• Design a Durable Key-Value Store — Start with a lookup table in one server's memory; end with a storage engine that survives crashes and power loss. One idea — a log file written before anything else moves — is the whole story, drawn and computed.
• Design S3-Like Object Storage — From one server writing files on disk to a store holding a million terabytes that survives an entire data centre dying — the bytes cut into recoverable pieces, spread across buildings, and repaired faster than they fail.
• Design a File-Sync Service — Drop a file in a folder and it appears, seconds later, on every device you own. Build the Dropbox shape from zero: cut files into content-hashed 4MB blocks so editing one block ships one block, guard a tiny metadata index with strong consistency while the bulky bytes stay loose, nudge devices with a held long-poll connection instead of a poll storm, and reconcile offline edits into first-writer-wins conflicts that never silently merge — with the delta-sync bandwidth save computed live.