RL Post-Training · Part 6

Beyond One Scalar Reward: Distillation, Effort Control, and Calibration

The 2025-2026 paradigms that change what the training signal is: dense on-policy distillation, multi-teacher integration, length and effort as trained targets, calibration-aware rewards, and the RLCD claims.

21On-Policy Distillation as Dense-Reward RL: Per-Token Reverse KL, Cost Arithmetic, and When It Beats RL50 min22Capability Integration: Mixed RL, Cascade RL, Parameter Merging, and Multi-Teacher On-Policy Distillation (MOPD)45 min23Length and Reasoning-Effort Control: Length Rewards, Token Budgets, Toggle, and Effort-Conditioned Experts50 min24Calibration-Aware RL: Proper Scoring Rules, the RLCR Reward, and Why Binary Rewards Teach Guessing50 min25The Name RLCD: What TypeSafe AI Claims, What Is Public, and the OpenJev Reconstruction50 min