Self-Distillation Enables Continual Learning

2026/01/27 by Idan Shenfeld, Mehul Damani, Jonas Hübotter +1 · 12 voices · 14 citations
#cs.LG

paper · pdf

Abstract

Continual learning, enabling models to acquire new skills and knowledge without degrading existing capabilities, remains a fundamental challenge for foundation models. While on-policy reinforcement learning can reduce forgetting, it requires explicit reward functions that are often unavailable. Learning from expert demonstrations, the primary alternative, is dominated by supervised fine-tuning (SFT), which is inherently off-policy. We introduce Self-Distillation Fine-Tuning (SDFT), a simple method that enables on-policy learning directly from demonstrations. SDFT leverages in-context learning by using a demonstration-conditioned model as its own teacher, generating on-policy training signals that preserve prior capabilities while acquiring new skills. Across skill learning and knowledge acquisition tasks, SDFT consistently outperforms SFT, achieving higher new-task accuracy while substantially reducing catastrophic forgetting. In sequential learning experiments, SDFT enables a single model to accumulate multiple skills over time without performance regression, establishing on-policy distillation as a practical path to continual learning from demonstrations.

Citations

• Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoning
• SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
• Affective Coherence Monitoring for Transformer-Based Language Models
• Training language models to follow instructions with human feedback
• Learning to summarize from human feedback
• Language Models are Few-Shot Learners
• A survey of inverse reinforcement learning: Challenges, methods and progress
• Distilling the Knowledge in a Neural Network

Cited by

• RP-OPSD: Resolution-Privileged On-Policy Self-Distillation for Multimodal Large Language Models
• Cura 1T: Specialized Model for Agentic Healthcare
• TSRouter: Dynamic Modality-Model Selection for Time Series Reasoning
• UCOB: Learning to Utilize and Evolve Agentic Skills via Credit-Aware On-Policy Bidirectional Self-Distillation
• Beyond Entropy: Correctness-Aware Advantage Shaping via Contrastive Policy Optimization
• Embarrassingly Simple Self-Distillation Improves Code Generation
• Position: Modular Memory is the Key to Continual Learning Agents
• D-OPSD: On-Policy Self-Distillation for Continuously Tuning Step-Distilled Diffusion Models
• Reinforcement Learning via Self-Distillation
• Masked Distillation: Internalizing the Chain-of-Thought in Language Models
• Aligning LLMs with Biomedical Knowledge using Balanced Fine-Tuning
• Weak-to-Strong On-Policy Distillation
• A Model Can Help Itself: Reward-Free Self-Training for LLM Reasoning
• Learning a Dense Reasoning Reward Model from Expert Demonstration via Inverse Reinforcement Learning
• On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters
• Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation
• Lightning OPD 2.0: Mitigating Style Bias in Cross-Teacher On-Policy Distillation for Large Reasoning Models

Discussions

• Self-Distillation Enables Continual Learning [pdf] [hn, 109 points, 26 comments]
• Self-Distillation Enables Continual Learning [hn, 2 points, 0 comments]
• Self-Distillation Enables Continual Learning [pdf] [bsky, 0 points, 0 comments]
• Self-Distillation Enables Continual Learning [PDF] https://arxiv.org/abs/2601.19897 [bsky, 0 points, 0 comments]
• Self-Distillation Enables Continual Learning [pdf] https://arxiv.org/abs/2601.19897 [comments] [63 points] [bsky, 0 points, 0 comments]
• https://bsky.app/profile/news.ycombinator.com.web.brid.gy/post/3mlzfj4geluj2 [bsky, 0 points, 0 comments]
• Self-Distillation Enables Continual Learning [pdf] https://arxiv.org/abs/2601.19897 (https://news.ycombinator.com/item?id=48165265) [bsky, 0 points, 0 comments]
• Self-distillation as continual learning feels backwards – teaching a model from its own outputs. But the math checks out for avoiding catastrophic forgetting. https://arxiv.org/abs/2601.19897 [bsky, 0 points, 1 comments]
• 今日読んだ論文 Self-Distillation Enables Continual Learning arxiv.org/abs/2601.19897 デモンストレーションのような例示をコンテキストに追加した教師を使って生徒に蒸留していく。やはりこういう方針が有望に感じられる。モデルサイズに影響されるのは当たり前だけどちょっと残念な要素でもあるかもしれない。大きいモデルを使わないといけないとし [bsky, 0 points, 0 comments]
• Self-Distillation Enables Continual Learning [pdf] https:// arxiv.org/abs/2601.19897 # arxiv [mastodon, 0 points, 0 comments]
• 📰 Self-distillation enables continual learning by refining models through iterative refinement and adaptation to new data, enhancing their ability to learn from experience without requiring retrainin [bsky, 0 points, 0 comments]
• Self-Distillation Enables Continual Learning [pdf] https://arxiv.org/abs/2601.19897 (https://news.ycombinator.com/item?id=48165265) [bsky, 0 points, 0 comments]

Related

• Continual Distillation Learning: Knowledge Distillation in Prompt-based Continual Learning
• Self Regulated Learning Mechanism for Data Efficient Knowledge\n Distillation
• SkillFactory: Self-Distillation For Learning Cognitive Behaviors
• Reinforcement Learning via Self-Distillation
• Understanding the Gains from Repeated Self-Distillation