MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent

2025/07/03 by Hongli Yu, Yu, Hongli, Tinghong Chen +19 · 45 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #cs.AI #cs.CL #cs.LG

paper · pdf · doi:10.48550/arxiv.2507.02259

published as International Conference on Learning Representations (ICLR), 2026 · Accepted to ICLR 2026 as an Oral presentation. OpenReview: https://openreview.net/forum?id=k5nIOvYGCL Project page: https://memagent-sialab.github.io/

arxiv created 2026/07/29 · arxiv updated 2026/07/30

Abstract

Despite improvements by length extrapolation, efficient attention and memory modules, handling infinitely long documents with linear complexity without performance degradation during extrapolation remains the ultimate challenge in long-text processing. We directly optimize for long-text tasks in an end-to-end fashion and introduce a novel agent workflow, MemAgent, which reads text in segments and updates the memory using an overwrite strategy. We extend the DAPO algorithm to facilitate training via independent-context multi-conversation generation. MemAgent has demonstrated superb long-context capabilities, being able to extrapolate from an 8K context trained on 32K text to a 3.5M QA task with performance loss < 5% and achieves 95%+ in 512K RULER test.

Citations

• Memp: Exploring Agent Procedural Memory
• MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents
• QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learning
• Group-in-Group Policy Optimization for LLM Agent Training
• Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
• RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning
• Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learning
• Understanding R1-Zero-Like Training: A Critical Perspective
• DAPO: An Open-Source LLM Reinforcement Learning System at Scale
• Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
• MoBA: Mixture of Block Attention for Long-Context LLMs
• Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
• Qwen2.5-1M Technical Report
• Kimi k1.5: Scaling Reinforcement Learning with LLMs
• NExtLong: Toward Effective Long-Context Training without Long Documents
• MiniMax-01: Scaling Foundation Models with Lightning Attention
• Titans: Learning to Memorize at Test Time
• Qwen2.5 Technical Report
• HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly
• E2LLM: Encoder Elongated Large Language Models for Long-Context Understanding and Reasoning
• FocusLLM: Precise Understanding of Long Context by Dynamic Condensing
• Attention as an RNN
• DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
• Test-Time Training on Graphs with Large Language Models (LLMs)
• RULER: What's the Real Context Size of Your Long-Context Language Models?
• Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models
• Training-Free Long-Context Scaling of Large Language Models
• BioXP-0.5B: Explainable Medical-AI via RL-GRPO
• Mamba: Linear-Time Sequence Modeling with Selective State Spaces
• LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models
• Compressing Context to Enhance Inference Efficiency of Large Language Models
• Scaling Laws of RoPE-based Extrapolation
• Efficient Streaming Language Models with Attention Sinks
• Effective Long-Context Scaling of Foundation Models
• YaRN: Efficient Context Window Extension of Large Language Models
• LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding
• MemoChat: Tuning LLMs to Use Memos for Consistent Long-Range Open-Domain Conversation
• Extending Context Window of Large Language Models via Positional Interpolation
• RET-LLM: Towards a General Read-Write Memory for Large Language Models
• RWKV: Reinventing RNNs for the Transformer Era
• MemoryBank: Enhancing Large Language Models with Long-Term Memory
• Scaling Transformer to 1M tokens and beyond with RMT
• Affective Coherence Monitoring for Transformer-Based Language Models
• Training language models to follow instructions with human feedback
• Efficiently Modeling Long Sequences with Structured State Spaces
• MuSiQue: Multihop Questions via Single-hop Question Composition
• A Dataset of Information-Seeking Questions and Answers Anchored in\n Research Papers
• RoFormer: Enhanced Transformer with Rotary Position Embedding
• Efficient Attentions for Long Document Summarization
• QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization
• Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps
• Memformer: A Memory-Augmented Transformer for Sequence Modeling
• Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
• Longformer: The Long-Document Transformer
• Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection
• Generating Long Sequences with Sparse Transformers
• HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
• The NarrativeQA Reading Comprehension Challenge
• Proximal Policy Optimization Algorithms
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• High-Dimensional Continuous Control Using Generalized Advantage\n Estimation
• Neural Turing Machines
• Memory Networks
• Long Short-Term Memory
• REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization

Cited by

• Agentic Entropy-Balanced Policy Optimization
• Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memory
• ACM: Agentic Context Management for Long Horizon Tasks
• Libra: Taming Attention Workload Skew in Long-Context LLM Training with Bounded Sequence Pool
• Beyond Heuristics: A Decision-Theoretic Framework for Agent Memory Management
• ABBEL: Learning Natural-Language Belief States for Memory-Efficient Interaction
• Memory-T1: Reinforcement Learning for Temporal Reasoning in Multi-session Agents
• Learning Hierarchical Procedural Memory for LLM Agents through Bayesian Selection and Contrastive Refinement
• Rhea: Role-aware Heuristic Episodic Attention for Conversational LLMs
• PersonaMem-v2: Towards Personalized Intelligence via Learning Implicit User Personas and Agentic Memory
• Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory
• General Agentic Memory Via Deep Research
• SkyRL-Agent: Efficient RL Training for Multi-turn LLM Agent
• Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning
• From Experience to Strategy: Empowering LLM Agents with Trainable Graph Memory
• IterResearch: Rethinking Long-Horizon Agents via Markovian State Reconstruction
• ContextPilot: Fast Long-Context Inference via Context Reuse
• MemSearcher: Training LLMs to Reason, Search and Manage Memory via End-to-End Reinforcement Learning
• MICA: Multi-granularity Intertemporal Credit Assignment for Long-Horizon Emotional Support Dialogue
• RGMem: Renormalization Group-inspired Memory Evolution for Language Agents
• AgentFold: Long-Horizon Web Agents with Proactive Context Management
• PaperAsk: A Benchmark for Reliability Evaluation of LLMs in Paper Search and Reading
• Memory as Action: Autonomous Context Curation for Long-Horizon Agentic Tasks
• Scaling Long-Horizon LLM Agent via Context-Folding
• AssoMem: Scalable Memory QA with Multi-Signal Associative Retrieval
• SkillOS: Learning Skill Curation for Self-Evolving Agents
• Demystifying Reinforcement Learning for Long-Horizon Tool-Using Agents: A Comprehensive Recipe
• Scaling LLM Multi-turn RL with End-to-end Summarization-based Context Management
• The Cognitive Bandwidth Bottleneck: Shifting Long-Horizon Agent from Planning with Actions to Planning with Schemas
• COSMIR: Chain Orchestrated Structured Memory for Iterative Reasoning over Long Context
• Mem-α: Learning Memory Construction via Reinforcement Learning
• ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory
• MemGen: Weaving Generative Latent Memory for Self-Evolving Agents
• Look Back to Reason Forward: Revisitable Memory for Long-Context LLM Agents
• ToMPO: Training LLM Strategic Decision Making from a Multi-Agent Perspective
• Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle
• xOffense: An AI-driven autonomous penetration testing framework with offensive knowledge-enhanced LLMs and multi agent systems
• WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learning
• ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarization
• OpenHA: A Series of Open-Source Hierarchical Agentic Models in Minecraft
• Memp: Exploring Agent Procedural Memory
• Sculptor: Empowering LLMs with Cognitive Agency via Active Context Management
• Git Context Controller: Manage the Context of LLM-based Agents like Git
• Are We Ready For An Agent-Native Memory System?
• Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions

Related

• MemGPT: Towards LLMs as Operating Systems
• LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration
• Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaboration
• UIO-LLMs: Unbiased Incremental Optimization for Long-Context LLMs
• From 128K to 4M: Efficient Training of Ultra-Long Context Large Language Models