What Should a Self-Teacher See? Privileged Context Design for On-Policy Self-Distillation

T1Covered by 1 source · first reported by arXiv at 24 Sept, 04:00 UTC

arXiv:2609.25623v1 Announce Type: cross Abstract: More privileged information does not always make a better teacher. We study this tension in on-policy self-distillation (OPSD), where a frozen copy of the base model scores the student's own rollouts under privileged context, conventionally a…

science

Sources

• T1What Should a Self-Teacher See? Privileged Context Design for On-Policy Self-DistillationarXiv24 Sept, 04:00