Post-Training

Turning a base model into an assistant after pretraining: reward signals, group-relative advantages, and the RL clusters — rollout, reference, reward, and the learner — that keep a policy, its sampler, and its judges in sync.

Explainers

• GRPO Advantage: Z-Score Your Siblings, Line by Line — PPO learns a whole second neural network just to guess how good an answer is. GRPO throws that away: sample a group of answers to the same prompt, and each answer's advantage is just how far above or below its siblings it scored. We walk the real tunix advantage estimators — GRPO's z-score, Dr.GRPO's un-normalized fix, and RLOO's leave-one-out baseline — three self-contained functions, computed live side by side.
• The RL Cluster: Five Roles, One Mesh Dial — An RL policy under training needs five jobs sharing one accelerator fleet: sample completions, score them, hold a frozen baseline, and push a gradient update back — and one line of config decides whether those jobs share memory or ship weights across the wire. We open Google's real tunix RL cluster to see exactly how, and compute what a weight sync costs either way.