Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training
2026/07/01 by Zijian Zhang, Rizhen Hu, Athanasios Glentis +4 · 17 voices
#cs.LG #cs.CL
paper · pdf
Abstract
Reinforcement learning (RL) has become a central component of post-training large language models (LLMs), yet little is understood about how RL adaptation is distributed across transformer layers. Existing approaches typically update all model parameters uniformly, implicitly assuming that every layer contributes similarly to the gains obtained during RL post-training. In this work, we challenge this assumption through a systematic layer-wise study of RL training. Surprisingly, we find that training a single transformer layer can recover most of the gains achieved by full-parameter RL training, and in some cases even surpass it. To quantify this phenomenon, we introduce the quantity layer contribution, which measures the fraction of full RL improvement recovered by training a layer in isolation. Across seven models spanning two model families (Qwen3, Qwen2.5), three RL algorithms (GRPO, GiGPO, Dr. GRPO), and multiple task domains including mathematical reasoning, code generation, and agentic decision-making, we observe a remarkably stable pattern: RL gains are highly concentrated in a small subset of, and in many cases even a single, transformer layers. More strikingly, the same structural pattern consistently emerges: high-contribution layers concentrate in the middle of the transformer stack, while layers near the input and output ends contribute substantially less. The resulting layer rankings remain strongly correlated across datasets, tasks, model families, and RL algorithms.
Citations
Discussions
- Is One Layer Enough? A Single Transformer Layer Matches Full-Parameter RL Train [hn, 153 points, 43 comments]
- big if true https://arxiv.org/abs/2607.01232 [bsky, 7 points, 0 comments]
- Papers I'm reading 📄 Is One Layer Enough? Training a single middle transformer layer can sometimes recover 100%+ of full-parameter RL gains. arxiv.org/abs/2607.01232 🔬 Program-as-Weights: define "fu [bsky, 2 points, 0 comments]
- This flips a common assumption: that RL post-training benefits from distributed updates across all layers. If one layer can absorb the adaptation signal, most parameters are just passengers. http://ar [bsky, 1 points, 0 comments]
- Is One Layer Enough? A Single Transformer Layer Matches Full-Parameter RL Train [bsky, 0 points, 0 comments]
- Is One Layer Enough? A Single Transformer Layer Matches Full-Parameter RL Train #HackerNews https://arxiv.org/abs/2607.01232 [bsky, 0 points, 0 comments]
- Is One Layer Enough? A Single Transformer Layer Matches Full-Parameter RL Train https://arxiv.org/abs/2607.01232 https://news.ycombinator.com/item?id=48760201 [bsky, 0 points, 0 comments]
- Is One Layer Enough? A Single Transformer Layer Matches Full-Parameter RL Train https://arxiv.org/abs/2607.01232 [bsky, 0 points, 0 comments]
- [19/30] 224 Upvotes, 43 Comments, 2 Posts, arXiv:2607.01232 🆕Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training Zijian Zhang, Rizhen Hu, Athanasios Glentis, [bsky, 0 points, 1 comments]
- A study reveals that training one transformer layer can achieve most gains from full-parameter RL, indicating that benefits are concentrated in a few key layers. https://arxiv.org/abs/2607.01232 [bsky, 0 points, 0 comments]
- https://bsky.app/profile/news.ycombinator.com.web.brid.gy/post/3mpojbsn6zoj2 [bsky, 0 points, 0 comments]
- How many layers are you on? arxiv.org/abs/2607.01232 [bsky, 0 points, 0 comments]
- Single layer matching full parameter performance? Either our models are massively over-parameterized or we're missing something about depth vs width. https://arxiv.org/abs/2607.01232 [bsky, 0 points, 1 comments]
- Is One Layer Enough? A Single Transformer Layer Matches Full-Parameter RL Train https:// arxiv.org/abs/2607.01232 # arxiv [mastodon, 0 points, 0 comments]
- One fine-tuned layer can match full RL post-training on LLMs, a paper says. Submit to upvote on HN https://news.ycombinator.com/submitlink?u=https%3A%2F%2Farxiv.org%2Fabs%2F2607.01232&t=Is%20One%20Lay [bsky, 0 points, 0 comments]
- Is One Layer Enough? A Single Transformer Layer Matches Full-Parameter RL Train https://arxiv.org/abs/2607.01232 (https://news.ycombinator.com/item?id=48760201) [bsky, 0 points, 0 comments]
- Is One Layer Enough? A Single Transformer Layer Matches Full-Parameter RL Train https://arxiv.org/abs/2607.01232 (https://news.ycombinator.com/item?id=48760201) [bsky, 0 points, 0 comments]
Related