RL Post-Training · Part 4

Making RL Run: Dynamics, Numerical Mismatch, and Compute Scaling

What goes wrong during long RL runs (entropy collapse, length drift, probability mismatch between inference and training engines) and what scaling studies say about compute.

13Training Dynamics: Entropy Collapse, KL Drift, Length Growth, and the Controls That Work50 min14Training-Inference Mismatch: Why the Rollout Engine and the Trainer Disagree on Probabilities, and the Four Families of Fixes45 min15RL Compute Scaling: Sigmoidal Fits, the ScaleRL Recipe, and What Longer Runs Actually Buy45 min