教程 / 实战普通
On-Policy Distillation Works Better Without the Teacher
内容摘要
A new Purdue paper analyzed teacher supervision in on-policy distillation. The teacher scores are mostly noise, and a fixed negative penalty beats a full teacher model on AIME24.
综合分类、实体标签与信息来源匹配