Applied Systems

The production plumbing that puts models in front of traffic: feature stores, serving paths, and the data systems that keep training and inference reading the same values under real load.

Explainers

• Design an Embedding Retrieval System — Search where the query and the item share no words at all — a photo finds the listing, a phrase finds the video, a person finds the people they might know. The retrieval shape under visual search, video search, similar listings, and People-You-May-Know, built from zero: what an embedding is (a learned point in space; nearness means similarity), why two towers instead of one model, a commit-first envelope on a 100M-item index, the honest deliberation between a brute-force GPU scan, IVF partitions, and an HNSW graph — with recall vs latency computed live on a real toy corpus — then the offline indexing pipeline, the online path, keeping the index fresh, training the towers on engagement with in-batch negatives, the multimodal shared space, and the failure sweep whose quietest box returns a confident 200 while silently retrieving nonsense.
• Design a Recommendation System (in the LLM Era) — Millions of items, one person, a hundred-millisecond budget — you can never score the whole catalog, so every large recommender is a funnel: cheap retrieval narrows millions to hundreds, a heavy ranker scores only the survivors, and a re-ranker shapes the final slate. We compute the scoring budget, dial an interactive funnel until it blows past 100ms, walk two-tower retrieval and the feature store behind the ranker, face the feedback loop that quietly collapses the catalog, and separate what the LLM era really changes from the hype — grounded in YouTube and Instagram's own papers.
• How to Design an ML System in 45 Minutes — An ML design prompt sounds like "build a model," but the room is grading something narrower: can you turn a fuzzy product goal into a crisp ML objective? That one translation is load-bearing — "reduce harmful content" can become a classifier with a review queue, a risk ranker for a bounded queue, or a rationale generator, and each spawns a different system. We make that fork the signature exhibit, then walk the rest of the round the way an interviewer scores it: where labels really come from, why the first model should be simple, why offline wins so often lose online, and the retraining loop that keeps a live model honest — grounded in Google's Rules of ML.
• Design a Ranking System — Ten million items, one person, two hundred milliseconds — you can never score the whole catalog with your best model, so every large ranker is a funnel. We compute the scoring budget first (a 5 ms/item heavy model against a 200 ms wall scores ~40 items, not ten million), then build the funnel live as a budget negotiation, walk candidate generation → light ranker → heavy ranker → re-ranking, and face the parts interviews skip: calibration (why an uncalibrated ad model loses real money), position bias in the training logs, offline-vs-online divergence, and the harmful-content moderation variant with its human-review queue — grounded in the published ad-CTR papers.
• ML Reliability in Production — When a web service breaks it throws a 500 and pages you; when a model breaks it keeps answering — confidently, wrongly, and silently. We name the four ways models rot (data drift, concept drift, upstream schema breaks, feature-pipeline rot), build a live drift simulator that computes PSI and KL off two real distributions and watches them trip their alarms at different speeds, then install the whole defense: monitoring the inputs and the outputs, evals on a schedule, the retrain-validate-shadow-promote loop, rollback when the model is stateful, and an incident runbook that triages bad model vs bad data vs bad code — drawn, computed, and animated.
• Design a RAG System — A model with a knowledge cutoff and no access to your private documents has to answer questions about both — so you retrieve the right passages and hand them to it as context. We size the corpus in bytes and vectors, dial an interactive chunker until retrieval hits, draw the vector index and reranker honestly, and end on the part everyone skips: how you measure whether any of it worked. Drawn, computed, and animated.
• Design a Feature Store — A model needs the same number at 3am training time and at the 50-millisecond serving moment — the user's 7-day click count had better mean the same thing in both. Build up from every team hand-rolling features in their service to a dual store fed by one definition: a columnar offline store holding years of history for point-in-time-correct training, a key-value online store holding the latest vector for 50K QPS under 50ms p99. Train/serve skew, the time-travel join, streaming freshness, and drift — drawn, computed, and animated.