MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learning

2025/06/17 by Tristan Tomilin, Tomilin, Tristan, Luka van den Boogaard +14 · 2 citations
Computer Science · #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2506.14990

Abstract

Benchmarks play a central role in reinforcement learning (RL) research, yet their computational constraints often shape what is studied. Despite the motivation of lifelong learning, most continual RL papers consider only 3-10 sequential tasks, as CPU-bound environments make longer sequences impractical. Meanwhile, continual learning in cooperative multi-agent settings remains largely unexplored. To address these gaps, we introduce MEAL (Multi-agent Environments for Adaptive Learning), the first benchmark for continual multi-agent RL. By leveraging JAX and GPU acceleration, MEAL enables training on sequences of 100 tasks in a few hours on a single GPU. We find that long task sequences reveal failure modes that do not appear at smaller scales.

Cited by

• Task diversity produces systematic transfer but inhibits continual reinforcement learning
• The Yokai Learning Environment: Tracking Beliefs Over Space and Time

Related

• JaxMARL: Multi-Agent RL Environments and Algorithms in JAX
• Multi-Agent Craftax: Benchmarking Open-Ended Multi-Agent Reinforcement Learning at the Hyperscale
• EARL: Efficient Agentic Reinforcement Learning Systems for Large Language Models
• Multi-Agent Reinforcement Learning: A Report on Challenges and Approaches
• WarpDrive: Extremely Fast End-to-End Deep Multi-Agent Reinforcement Learning on a GPU