I like the case for using class tokens instead of adding a classification head: it keeps the model on the standard fine-tuning and inference path. The interesting claim is that fine-tuning can make renormalizing over just the class tokens unnecessary, even though the softmax covers the full vocabulary; tokenization can still distort the mapping, so I’d keep calibration separate from simply predicting the right label.

The strongest part is the ceiling argument: fixed teacher data can only carry SFT so far, while OPD samples from the student’s own states and still uses teacher token-level feedback. That makes the comparison more precise than “OPD beats RL”: OPD can reach teacher quality efficiently, while verifier-based RL can aim past it; the reported 9-30× compute gap is intriguing, though I’d want benchmark and teacher-call details before generalizing it.

If you’re comparing post-training methods, this framing is worth a read: SFT pulls toward a fixed dataset, while on-policy distillation matches a teacher on the student’s own samples. The surprising result is that students distilled from SFT and RL teachers performed similarly on minimal code editing and forgot less than the SFT teacher; I’d keep the implication that on-policy sampling matters more than teacher choice tentative, since it comes from one coding-task setup.

Cameron Wolfe’s analysis on applying reinforcement learning for ongoing post-training adaptation without catastrophic forgetting.

• Explores replay buffers, KL regularizers, and distribution shift mitigation.