Self-Adapting Language Models

2025/06/12 by Adam Zweiger, Zweiger, Adam, Jyothish Pari +9 · 26 voices · 17 citations
Computer Science · Medicine · #Topic Modeling #Artificial Intelligence in Healthcare and Education #Domain Adaptation and Few-Shot Learning

paper · pdf · doi:10.48550/arxiv.2506.10943

Abstract

Large language models (LLMs) are powerful but static; they lack mechanisms to adapt their weights in response to new tasks, knowledge, or examples. We introduce Self-Adapting LLMs (SEAL), a framework that enables LLMs to self-adapt by generating their own finetuning data and update directives. Given a new input, the model produces a self-edit-a generation that may restructure the information in different ways, specify optimization hyperparameters, or invoke tools for data augmentation and gradient-based updates. Through supervised finetuning (SFT), these self-edits result in persistent weight updates, enabling lasting adaptation. To train the model to produce effective self-edits, we use a reinforcement learning loop with the downstream performance of the updated model as the reward signal. Unlike prior approaches that rely on separate adaptation modules or auxiliary networks, SEAL directly uses the model's own generation to control its adaptation process. Experiments on knowledge incorporation and few-shot generalization show that SEAL is a promising step toward language models capable of self-directed adaptation. Our website and code is available at https://jyopari.github.io/posts/seal.

Citations

• Reasoning to Learn from Latent Thoughts
• Transformer-Squared: Self-adaptive LLMs
• Self-Improvement in Language Models: The Sharpening Mechanism
• The Surprising Effectiveness of Test-Time Training for Few-Shot Learning
• Synthetic continued pretraining
• Physics of Language Models: Part 3.1, Knowledge Storage and Extraction
• Efficient Memory Management for Large Language Model Serving with PagedAttention
• TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
• Training language models to follow instructions with human feedback
• A Modern Self-Referential Weight Matrix That Learns to Modify Itself
• Language Models are Few-Shot Learners
• Learning to reinforcement learn
• RL²: Fast Reinforcement Learning via Slow Reinforcement Learning
• Distilling the Knowledge in a Neural Network

Cited by

• Discovery by Dreaming: Cross-Domain Recombination in Artificial Memory
• No Labels, No Problem: Training Visual Reasoners with Multimodal Verifiers
• Learning to Seek Evidence: A Verifiable Reasoning Agent with Causal Faithfulness Analysis
• Hail to the Thief: Exploring Attacks and Defenses in Decentralised GRPO
• Understanding LoRA as Knowledge Memory: An Empirical Analysis
• Teaching Models to Teach Themselves: Reasoning at the Edge of Learnability
• Self-evolving expertise in complex non-verifiable subject domains: dialogue as implicit meta-RL
• Self-Improving LLM Agents at Test-Time
• Learning on the Job: Test-Time Curricula for Targeted Reinforcement Learning
• Specialization after Generalization: Towards Understanding Test-Time Training in Foundation Models
• SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass
• SMARTER: A Data-efficient Framework to Improve Toxicity Detection with Explanation via Self-augmenting Large Language Models
• Synthetic bootstrapped pretraining
• Language Self-Play For Data-Free Training
• RL's Razor: Why Online Reinforcement Learning Forgets Less
• ALAS: Autonomous Learning Agent for Self-Updating Language Models
• Self-Questioning Language Models

Discussions

• Self-Adapting Language Models [hn, 246 points, 73 comments]
• arxiv.org/abs/2506.10943 "To train the model to produce effective self-edits, we use a reinforcement learning loop with the downstream performance of the updated model as the reward signal. ... direct [bsky, 8 points, 0 comments]
• What do you think of SEALs @eryk.bsky.social ? Is this the beginning of LLMs to have something like instrumental reason or just another data feedback loop? arxiv.org/abs/2506.10943 jyopari.github.io/p [bsky, 4 points, 1 comments]
• also, that’s another reference to the SEAL paper, i guess that means i have to read it now arxiv.org/abs/2506.10943 [bsky, 3 points, 0 comments]
• arxiv.org/pdf/2506.10943 [bsky, 2 points, 1 comments]
• Self adapting language models, still early but fascinating prospects. There's a dimensionality curse of course: since the dimensions the LLM can touch per generated token are very small as such, needs [bsky, 1 points, 1 comments]
• Well well well arxiv.org/abs/2506.10943 I explained the framework in advance with my language including the elevation of concepts for self learning they called downstream. With my Meta Verse, too. Obj [bsky, 1 points, 1 comments]
• oh spannend! 😃 "Self-Adapting Language Models" [bsky, 1 points, 0 comments]
• J'ai vu que cette nouvelle version reste très difficile pour eux, mais qu'il y a du progrès (ils sont désormais à 10%, contre 0% il y a quelques mois). Mais de nouvelles pistes arrivent sans cesse, cf [bsky, 1 points, 1 comments]
• This could be game changing arxiv.org/abs/2506.10943 [bsky, 1 points, 0 comments]
• Self-Adapting Language Models https://arxiv.org/abs/2506.10943 (http://news.ycombinator.com/item?id=44271284) [bsky, 0 points, 0 comments]
• "Self-Adapting Language Models" arxiv.org/abs/2506.10943 [bsky, 0 points, 0 comments]
• This is pretty crazy, i was hoping something like this would come out. The RL stuff is pretty gnarly and kind of assumed the ai would be better tuning it. arxiv.org/abs/2506.10943 [bsky, 0 points, 1 comments]
• https://arxiv.org/abs/2506.10943 本論文では、自己適応言語モデルについて解説されています。 言語モデルが自身の挙動を監視し、必要に応じて適応することで性能向上を目指します。 詳細な手法や実験結果については、リンク先の論文をご覧ください。 [bsky, 0 points, 0 comments]
• The RL loop is elegant - model generates "self-edits," applies them via LoRA updates, gets rewarded based on downstream performance. This could be huge as we hit data scaling limits. arxiv.org/pdf/250 [bsky, 0 points, 0 comments]
• Self-Adapting Language Models introduce SEAL, a framework where LLMs generate their own training data and update plans to improve knowledge integration and few-shot learning. A step toward autonomous [bsky, 0 points, 0 comments]
• ⚡ Hackernews Top story: Self-Adapting Language Models [bsky, 0 points, 0 comments]
• Self-Adapting Language Models https://arxiv.org/abs/2506.10943 (https://news.ycombinator.com/item?id=44271284) [bsky, 0 points, 0 comments]
• Self-Adapting Language Models https://arxiv.org/abs/2506.10943 (https://news.ycombinator.com/item?id=44271284) [bsky, 0 points, 0 comments]
• Self-Adapting Language Models [bsky, 0 points, 1 comments]
• Self-Adapting Language Models [bsky, 0 points, 0 comments]
• Self-Adapting Language Models #HackerNews https://arxiv.org/abs/2506.10943 [bsky, 0 points, 0 comments]
• Self-Adapting Language Models https://arxiv.org/abs/2506.10943 https://news.ycombinator.com/item?id=44271284 [bsky, 0 points, 0 comments]
• Self-Adapting Language Models https://arxiv.org/abs/2506.10943 [bsky, 0 points, 0 comments]
• Self-Adapting Language Models arxiv.org/abs/2506.10943 [bsky, 0 points, 0 comments]
• https://bsky.app/profile/news.ycombinator.com.web.brid.gy/post/3lrjknenhgko2 [bsky, 0 points, 0 comments]

Related

• Data Efficient Adaptation in Large Language Models via Continuous Low-Rank Fine-Tuning
• STABLE: Gated Continual Learning for Large Language Models
• SELF: Self-Evolution with Language Feedback
• RE-Adapt: Reverse Engineered Adaptation of Large Language Models
• Teaching Language Models to Self-Improve by Learning from Language Feedback