Do Language Models Need Sleep? Offline Recurrence for Improved Online Inference

2026/05/25 by Sangyun Lee, Sean McLeish, Tom Goldstein +1 · 24 voices
#cs.CL #cs.AI

paper · pdf

Abstract

Transformer-based large language models are increasingly used for long-horizon tasks; however, their attention mechanism scales poorly with context length. To handle this, we study a sleep-like consolidation mechanism in which a model periodically converts recent context into persistent fast weights before clearing its key-value cache. During sleep, the model performs N offline recurrent passes over the accumulated context and updates the fast weights in its state-space model (SSM) blocks through a learned local rule. During inference, this shifts extra computation to sleep while preserving the latency of wake-time prediction. We test our method on controlled synthetic tasks, including cellular automata and multi-hop graph retrieval, as well as a realistic math reasoning task, on which a regular transformer as well as SSM-attention hybrid models fail. We then show that increasing sleep duration N for our models improves performance, with the largest gains on examples that require deeper reasoning.

Citations

• Deep sequence models tend to memorize geometrically; it is unclear why
• Scaling Latent Reasoning via Looped Language Models
• Jet-Nemotron: Efficient Language Model with Post Neural Architecture Search
• Latent Iteration as Renormalization: Inference-Time Recurrence in a Depth-Recurrent Transformer Flows Attention Geometry Toward the SYK Conformal Fixed Point
• Transformers Learn Shortcuts to Automata
• Attention Is All You Need
• Vacunación del niño inmigrante y adoptado en España

Discussions

• A sleep-like consolidation mechanism for LLMs [hn, 212 points, 140 comments]
• arxiv.org/abs/2605.26099 Language Models Need Sleep "attention mechanism scales poorly with context length. To handle this, we study a sleep-like consolidation mechanism in which a model periodically [bsky, 15 points, 0 comments]
• incredible paper title [bsky, 2 points, 0 comments]
• LLMも人間同様に「休息」が必要？最新研究で、モデルを一時停止させて推論を繰り返すと、誤答率が大幅に低下することが判明。計算資源の最適化だけでなく、AIの思考プロセス改善という新たな視点が注目されます。効率的な運用が今後の鍵を握りそうです。 #AI #TechNews https://arxiv.org/abs/2605.26099 [bsky, 1 points, 0 comments]
• It makes sense to convert long memories into weights. That may be achieved in downtime (aka “sleep”) mode. arxiv.org/abs/2605.26099 [bsky, 1 points, 0 comments]
• Language Models Need Sleep https://arxiv.org/abs/2605.26099 https://news.ycombinator.com/item?id=48281226 [bsky, 1 points, 1 comments]
• They dream of electric shlop arxiv.org/abs/2605.26099 [bsky, 1 points, 0 comments]
• Language Models Need Sleep #HackerNews https://arxiv.org/abs/2605.26099 [bsky, 0 points, 0 comments]
• Language Models Need Sleep https://arxiv.org/abs/2605.26099 (https://news.ycombinator.com/item?id=48281226) [bsky, 0 points, 0 comments]
• A sleep-like consolidation mechanism for LLMs https:// arxiv.org/abs/2605.26099 # arxiv # llm # llms [mastodon, 0 points, 0 comments]
• Language Models Need Sleep https://arxiv.org/abs/2605.26099 [bsky, 0 points, 0 comments]
• 📰 A sleep-like consolidation mechanism for LLMs 🔗 https://arxiv.org/abs/2605.26099 💬 Discuss on HN [bsky, 0 points, 0 comments]
• [12/30] 275 Upvotes, 142 Comments, 3 Posts, arXiv:2605.26099 🆕Language Models Need Sleep Sangyun Lee, Sean McLeish, Tom Goldstein, Giulia Fanti [bsky, 0 points, 1 comments]
• A sleep-like consolidation mechanism for LLMs https://arxiv.org/abs/2605.26099 (http://news.ycombinator.com/item?id=48281226) [bsky, 0 points, 0 comments]
• Do Language Models Need Sleep? Offline Recurrence for Improved Online Inference arxiv.org/abs/2605.26099 [bsky, 0 points, 0 comments]
• arxiv.org/abs/2605.26099 [bsky, 0 points, 0 comments]
• 📰 Language models require sleep to function optimally and maintain their accuracy and efficiency in processing tasks. 🔗 https://arxiv.org/abs/2605.26099 #Tech #Dev [bsky, 0 points, 0 comments]
• bring out the claude claude's sleepin well i guess you're gonna have to go wake em up now aint ya arxiv.org/abs/2605.26099 [bsky, 0 points, 0 comments]
• Interesting idea, but I doubt it will help. The basic design is flawed, the concept is all wrong, and nothing can fix that. [bsky, 0 points, 0 comments]
• Researchers propose a sleep-like mechanism for large language models to consolidate recent context into persistent weights during offline computations, significantly improving performance on complex t [bsky, 0 points, 0 comments]
• Exciting advances in LLMs! A new study introduces a sleep-like consolidation mechanism, enhancing model performance by optimizing memory retention and reducing interference. This could redefine how we [bsky, 0 points, 0 comments]
• Language Models Need Sleep view on hacker news [bsky, 0 points, 0 comments]
• Language Models Need Sleep https://arxiv.org/abs/2605.26099 (https://news.ycombinator.com/item?id=48281226) [bsky, 0 points, 0 comments]
• Language Models Need Sleep [bsky, 0 points, 0 comments]

Related

• Sleep-time Compute: Beyond Inference Scaling at Test-time
• Star Attention: Efficient LLM Inference over Long Sequences
• Long Distance Relationships without Time Travel: Boosting the Performance of a Sparse Predictive Autoencoder in Sequence Modeling
• RepCNN: Micro-sized, Mighty Models for Wakeword Detection
• SHARP: Accelerating Language Model Inference by SHaring Adjacent layers with Recovery Parameters