---
title: OPSD (On-Policy Self-Distillation)
type: vocabulary
url: "https://www.envisioning.com/vocab/opsd-on-policy-self-distillation"
summary: Self-distillation variant where a single model acts as both teacher and student by conditioning on privileged context (e.g., verified reasoning traces) while the student sees only the question.
year: 2026
generality: 0.65
---

# OPSD (On-Policy Self-Distillation)

Self-distillation variant where a single model acts as both teacher and student by conditioning on privileged context (e.g., verified reasoning traces) while the student sees only the question.
On-Policy Self-Distillation (OPSD) is a learning algorithm in which a single language model is trained to act as both teacher and student under different contexts, eliminating the need for a separate, often larger, teacher model. The teacher policy conditions on privileged information — typically the ground-truth reasoning trace — while the student policy sees only the question. Training minimizes per-token KL divergence between these two distributions, computed over the student's own on-policy rollouts (not a static dataset), which addresses the train/inference distribution mismatch that plagues off-policy distillation. OPSD achieves higher token efficiency than reinforcement-learning methods (e.g., GRPO) and outperforms off-policy distillation because the student learns from its own sampling distribution.

OPSD belongs to the broader family of self-distillation techniques, where the supervisor and the learner share weights. It differs from classical knowledge-distillation (Hinton 2015) by removing the external teacher, and from earlier self-distillation variants (Furlanello 2018) by adding the privileged-context split and the on-policy rollouts. The technique is foundational for several 2026 self-improving reasoning methods, including test-time-policy-optimization (TTPO), which generalizes OPSD to the label-free setting by replacing privileged traces with majority-vote pseudo-labels and adding an asymmetric RL penalty for disagreeing rollouts.

The canonical paper is Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models by Zhao et al. (CMU), arXiv:2601.18734, January 2026. Code at github.com/siyan-zhao/OPSD. Subsequent work extends OPSD to diffusion models (arXiv:2608.24646), multilingual reasoning (arXiv:2605.09548), and visual contrastive settings (arXiv:2607.21556).

---
Source: Envisioning — Technology Research Institute (https://www.envisioning.com/vocab/opsd-on-policy-self-distillation)