Fundamentals

The architecture under every modern model, built from first principles: tokens and embeddings, attention computed number by number, multi-head and the KV cache, positional encoding, and the block stacked into a full network.

Explainers

• The Transformer, End to End — The researcher-round staple: draw the architecture from memory and explain every tensor. We build it once, honestly — token → embedding → attention computed on four real tokens (Q/K/V, a QKᵀ heatmap, softmax, the weighted sum, every number real) → multi-head → the KV cache and the MHA/MQA/GQA/MLA memory ladder dialed live → RoPE as a rotation → the MLP and residual highway → norm placement → the block stacked N times → a parameter count that reproduces Llama-3-8B to the last billion.