δ-mem: Efficient Online Memory for Large Language Models

2026/05/12 by Jingdi Lei, Di Zhang, Junxian Li +7 · 24 voices · 2 citations
#cs.AI

paper · pdf

Abstract

Large language models increasingly need to accumulate and reuse historical information in long-term assistants and agent systems. Simply expanding the context window is costly and often fails to ensure effective context utilization. We propose δ-mem, a lightweight memory mechanism that augments a frozen full-attention backbone with a compact online state of associative memory. δ-mem compresses past information into a fixed-size state matrix updated by delta-rule learning, and uses its readout to generate low-rank corrections to the backbone's attention computation during generation. With only an 8×8 online memory state, δ-mem improves the average score to 1.10× that of the frozen backbone and 1.15× that of the strongest non-δ-mem memory baseline. It achieves larger gains on memory-heavy benchmarks, reaching 1.31× on MemoryAgentBench and 1.20× on LoCoMo, while largely preserving general capabilities. These results show that effective memory can be realized through a compact online state directly coupled with attention computation, without full fine-tuning, backbone replacement, or explicit context extension.

Citations

• Kimi Linear: An Expressive, Efficient Attention Architecture
• LLMs Get Lost In Multi-Turn Conversation
• Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
• Titans: Learning to Memorize at Test Time
• MemGPT: Towards LLMs as Operating Systems
• Augmenting Language Models with Long-Term Memory
• Generative Agents: Interactive Simulacra of Human Behavior
• Memorizing Transformers

Cited by

• Metis: Memory Foundation Model
• Σ-Mem: An Online Reliability Memory for LLM-based Multi-Agent Systems

Discussions

• δ-mem: Efficient Online Memory for Large Language Models [hn, 240 points, 60 comments]
• The real bottleneck in long-context LLMs isn't memory size, it's memory *volatility* : how fast the model forgets which stored information is still relevant. Δ-Mem tackles that by treating memory upda [bsky, 7 points, 1 comments]
• Δ-Mem: Efficient Online Memory for Large Language Models https://arxiv.org/abs/2605.12357 https://news.ycombinator.com/item?id=48158506 [bsky, 1 points, 0 comments]
• arXiv Upvote Trends Top 30 [1/30] [2/30] [3/30] [4/30] [5/30] [6/30] [7/30] [8/30] [9/30] [10/30] [11/30] [12/30] [13/30] [14/30] [15/30] [16/30] [17/30] [18/30] [19/30] [20/30] [21/30] [22/30] [23/30 [bsky, 1 points, 0 comments]
• -mem: Efficient Online Memory for Large Language Models arxiv.org/abs/2605.12357 [bsky, 1 points, 0 comments]
• Δ-Mem: Efficient Online Memory for Large Language Models [bsky, 1 points, 0 comments]
• 2/2 ingenieros de IA que trabajan en la escalabilidad de aplicaciones en tiempo real. Fuente: https://arxiv.org/abs/2605.12357 [bsky, 0 points, 0 comments]
• Δ-Mem: Efficient Online Memory for Large Language Models #HackerNews https://arxiv.org/abs/2605.12357 [bsky, 0 points, 0 comments]
• δ-mem: Efficient Online Memory for Large Language Models https://arxiv.org/abs/2605.12357 https://news.ycombinator.com/item?id=48158506 [bsky, 0 points, 0 comments]
• Δ-Mem: Efficient Online Memory for Large Language Models https://arxiv.org/abs/2605.12357 [bsky, 0 points, 0 comments]
• δ-mem: Efficient Online Memory for Large Language Models View Article | Join the HN Conversation Summary of HN discussion 🧵👇 [bsky, 0 points, 1 comments]
• δ-mem: Efficient Online Memory for Large Language Models https://arxiv.org/abs/2605.12357 [comments] [210 points] [bsky, 0 points, 0 comments]
• 📰 δ-mem: Efficient Online Memory for Large Language Models 🔗 https://arxiv.org/abs/2605.12357 💬 Discuss on HN [bsky, 0 points, 0 comments]
• https://bsky.app/profile/news.ycombinator.com.web.brid.gy/post/3mlxu4u334ar2 [bsky, 0 points, 0 comments]
• δ-mem: Efficient Online Memory for Large Language Models https://arxiv.org/abs/2605.12357 (http://news.ycombinator.com/item?id=48158506) [bsky, 0 points, 0 comments]
• 📰 Δ-Mem: A novel online memory technique for large language models enhances efficiency and performance by leveraging advanced data structures and algorithms. 🔗 https://arxiv.org/abs/2605.12357 #Tech [bsky, 0 points, 0 comments]
• LLMのメモリ効率を劇的に改善する新技術「Δ-Mem」が登場。計算コストを抑えつつ、リアルタイムでの学習や記憶の更新を可能にします。長文コンテキスト処理やパーソナライズ化の課題を解決する重要なブレイクスルーとなりそうです。 #AI #TechNews https://arxiv.org/abs/2605.12357 [bsky, 0 points, 0 comments]
• Δ-Mem: Efficient Online Memory for Large Language Models view on hacker news [bsky, 0 points, 0 comments]
• δ-mem: Efficient Online Memory for Large Language Models https:// arxiv.org/abs/2605.12357 # arxiv [mastodon, 0 points, 0 comments]
• Introducing Δ-Mem: a groundbreaking approach to enhance online memory for large language models. This innovation optimizes memory usage, boosting efficiency and performance. Discover how Δ-Mem could r [bsky, 0 points, 0 comments]
• δ-mem: add a tiny 8×8 online associative memory to a frozen full-attention transformer. Update it with a delta-rule, read it out to generate low-rank attention corrections. Reports ~1.10× avg gain; bi [bsky, 0 points, 0 comments]
• Δ-Mem: Efficient Online Memory for Large Language Models https://arxiv.org/abs/2605.12357 (https://news.ycombinator.com/item?id=48158506) [bsky, 0 points, 0 comments]
• Δ-Mem: Efficient Online Memory for Large Language Models https://arxiv.org/abs/2605.12357 (https://news.ycombinator.com/item?id=48158506) [bsky, 0 points, 0 comments]
• [2605.12357] $δ$-mem: Efficient Online Memory for Large Language Models — A practical arXiv read on δ-mem—an approach to adding efficient online memory to LLMs without dragging down latency or compute [bsky, 0 points, 0 comments]

Related

• LightMem: Lightweight and Efficient Memory-Augmented Generation
• Mem-α: Learning Memory Construction via Reinforcement Learning
• ABC: Attention with Bounded-memory Control
• MemMamba: Rethinking Memory Patterns in State Space Model
• Simple linear attention language models balance the recall-throughput tradeoff