How do you approach a machine learning system design interview?

Start at the goal, not the model. The interviewer is grading whether you can turn a fuzzy product goal into a crisp ML objective — the one step most candidates skip. A workable 45-minute split is: ~5 minutes on the business objective (including whether a heuristic beats ML for now), 5–10 on translating that goal into an ML objective and naming one alternative framing you passed on, ~10 on data and features (where labels come from, their delay and bias, and the train/serve-skew guard), ~10 on a simple-first model and the offline-vs-online metric, ~10 on serving and the drift-and-retrain loop, and ~5 to wrap. The translation deserves as much time as the model and metrics combined.

What is the difference between a business objective and an ML objective?

The business objective is the product outcome you want — reduce harmful content, keep people watching, cut fraud. An ML objective is a specific thing a specific model predicts, with a specific loss and a specific label. The load-bearing insight is that one business objective admits several honest ML translations, each of which spawns a different system. "Reduce harmful content" can become classification (predict the probability a post violates policy, block above a threshold), ranking (predict a risk score that just orders a fixed-capacity review queue), or generation (predict the rationale a moderator would write). A classifier, a ranker, and a fine-tuned language model are three different systems with three different data needs — so choosing among them, out loud and for a reason, is the interview.

Where do the labels come from in an ML system?

Three honest sources, each with a catch. Natural labels are signals the product already generates — clicks, purchases, watch time, chargebacks — cheap and plentiful, but biased by what the system already showed and sometimes arriving weeks late. Human annotation is moderators or raters marking examples by hand — accurate but expensive and slow, so you get thousands not billions, which argues for a simpler model; it is also often censored, since a fraud system only sees chargebacks for transactions it let through. Weak supervision generates noisy labels programmatically from heuristics or a distant signal, trading accuracy for volume to bootstrap a first labeled set. Two failure modes hide here: label delay leaves the model structurally behind the world, and the feedback loop lets the model shape its own future training data.

Why does an ML model win offline but lose online?

Offline metrics like AUC and precision are computed on a held-out slice of logged data; online metrics like watch time and revenue are measured on live users in an A/B test. The offline number is a cheap proxy for the online one, and it can move the opposite way. The main reason: the offline log was written by the current model, so it only contains items the old system chose to show — a higher offline score often just means "agrees more with the model that generated the log," not "serves users better." Optimizing a proxy too hard also finds where the proxy and the goal diverge: a click model learns clickbait, a watch-time model learns autoplay traps. The habit that signals maturity is to quote the offline number, then name the online metric you would actually ship on and why the two can diverge.

Should you use deep learning or a simple model in an ML interview?

Start simple and make the interviewer watch you resist the shiny thing — Google's Rules of ML put it as "the first model provides the biggest boost to your product, so it doesn't need to be fancy." For most tabular problems the first model is logistic regression or gradient-boosted trees: they train in seconds, need only thousands of labels, are interpretable when they misbehave, and are cheap to serve. As a rough heuristic, a linear model fits well with about ten examples per feature, while a deep net needs orders of magnitude more labels before it stops overfitting, so the empirical cross-over where deep reliably wins usually sits around 100,000 to a million labeled examples. Embeddings earn their keep when the signal lives in relationships you cannot hand-write as features — the meaning of text, similarity between images, affinity you never described.

How to Design an ML System in 45 Minutes

An ML design prompt sounds like "build a model," but the room is grading something narrower: can you turn a fuzzy product goal into a crisp ML objective? That one translation is load-bearing — "reduce harmful content" can become a classifier with a review queue, a risk ranker for a bounded queue, or a rationale generator, and each spawns a different system. We make that fork the signature exhibit, then walk the rest of the round the way an interviewer scores it: where labels really come from, why the first model should be simple, why offline wins so often lose online, and the retraining loop that keeps a live model honest — grounded in Google's Rules of ML.

Concept · AI / ML. The source ↗

A free, interactive, animated visual explainer of How to Design an ML System in 45 Minutes — built to be understood, not skimmed.

Questions

How do you approach a machine learning system design interview?: Start at the goal, not the model. The interviewer is grading whether you can turn a fuzzy product goal into a crisp ML objective — the one step most candidates skip. A workable 45-minute split is: ~5 minutes on the business objective (including whether a heuristic beats ML for now), 5–10 on translating that goal into an ML objective and naming one alternative framing you passed on, ~10 on data and features (where labels come from, their delay and bias, and the train/serve-skew guard), ~10 on a simple-first model and the offline-vs-online metric, ~10 on serving and the drift-and-retrain loop, and ~5 to wrap. The translation deserves as much time as the model and metrics combined.
What is the difference between a business objective and an ML objective?: The business objective is the product outcome you want — reduce harmful content, keep people watching, cut fraud. An ML objective is a specific thing a specific model predicts, with a specific loss and a specific label. The load-bearing insight is that one business objective admits several honest ML translations, each of which spawns a different system. "Reduce harmful content" can become classification (predict the probability a post violates policy, block above a threshold), ranking (predict a risk score that just orders a fixed-capacity review queue), or generation (predict the rationale a moderator would write). A classifier, a ranker, and a fine-tuned language model are three different systems with three different data needs — so choosing among them, out loud and for a reason, is the interview.
Where do the labels come from in an ML system?: Three honest sources, each with a catch. Natural labels are signals the product already generates — clicks, purchases, watch time, chargebacks — cheap and plentiful, but biased by what the system already showed and sometimes arriving weeks late. Human annotation is moderators or raters marking examples by hand — accurate but expensive and slow, so you get thousands not billions, which argues for a simpler model; it is also often censored, since a fraud system only sees chargebacks for transactions it let through. Weak supervision generates noisy labels programmatically from heuristics or a distant signal, trading accuracy for volume to bootstrap a first labeled set. Two failure modes hide here: label delay leaves the model structurally behind the world, and the feedback loop lets the model shape its own future training data.
Why does an ML model win offline but lose online?: Offline metrics like AUC and precision are computed on a held-out slice of logged data; online metrics like watch time and revenue are measured on live users in an A/B test. The offline number is a cheap proxy for the online one, and it can move the opposite way. The main reason: the offline log was written by the current model, so it only contains items the old system chose to show — a higher offline score often just means "agrees more with the model that generated the log," not "serves users better." Optimizing a proxy too hard also finds where the proxy and the goal diverge: a click model learns clickbait, a watch-time model learns autoplay traps. The habit that signals maturity is to quote the offline number, then name the online metric you would actually ship on and why the two can diverge.
Should you use deep learning or a simple model in an ML interview?: Start simple and make the interviewer watch you resist the shiny thing — Google's Rules of ML put it as "the first model provides the biggest boost to your product, so it doesn't need to be fancy." For most tabular problems the first model is logistic regression or gradient-boosted trees: they train in seconds, need only thousands of labels, are interpretable when they misbehave, and are cheap to serve. As a rough heuristic, a linear model fits well with about ten examples per feature, while a deep net needs orders of magnitude more labels before it stops overfitting, so the empirical cross-over where deep reliably wins usually sits around 100,000 to a million labeled examples. Embeddings earn their keep when the signal lives in relationships you cannot hand-write as features — the meaning of text, similarity between images, affinity you never described.

How to Design an ML System in 45 Minutes

Questions

Related explainers