RL Post-Training · Part 7

Case Studies: Documented Post-Training Recipes, 2024-2026

Seven recipes read stage by stage from their primary technical reports, showing how the mechanisms of Parts 1-6 were composed, with numbers, orderings and what each report says went wrong.

26Case Study: Llama 3 and Tulu 3 (2024) - DPO-First Post-Training, the PPO Comparison, and the Birth of RLVR50 min27Case Study: DeepSeek-R1 - R1-Zero, the Four-Stage R1 Pipeline, Its Compute, and What the Nature Version Changed45 min28Case Study: Kimi k1.5 to K3 - Mirror-Descent Policy Optimization, Self-Critic Rewards, Token-Budget RL, and Nine-Expert MOPD45 min29Case Study: Qwen3 - Four-Stage Reasoning Pipeline, 3,995 Query-Verifier Pairs, Thinking-Mode Fusion, and Strong-to-Weak Distillation40 min30Case Study: DeepSeek-V3.2 to V4 - Scaling GRPO Past 10% of Pre-Training Compute, Then Replacing Mixed RL with On-Policy Distillation40 min31Case Study: Olmo 3 - OlmoRL, RL-Zero from a Fully Open Base, and Measured Rollout Economics45 min32Case Study: GLM-5 and Nemotron 3 Ultra - Asynchronous Agentic RL, Deterministic Indexer Top-k, Environments the Report Calls Open, and the Limits of MOPD50 min