Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks

2020/06/14 by Georgios Papoudakis, Papoudakis, Georgios, Filippos Christianos +5 · 52 citations
Computer Science · Mathematics · #Advanced Multi-Objective Optimization Algorithms #Artificial Intelligence (cs.AI) #Evolutionary Algorithms and Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Multiagent Systems (cs.MA) #Open Source Software Innovations #Reinforcement Learning in Robotics #cs.AI #cs.LG #cs.MA #stat.ML

paper · pdf · doi:10.48550/arxiv.2006.07869

Published in 35th Conference on Neural Information Processing Systems (NeurIPS 2021) Track on Datasets and Benchmarks

openalex publication_date 2020/06/14 · arxiv created 2021/11/09 · arxiv updated 2021/11/10 · openalex created_date 2022/07/26 · openalex updated_date 2026/07/28

Abstract

Multi-agent deep reinforcement learning (MARL) suffers from a lack of commonly-used evaluation tasks and criteria, making comparisons between approaches difficult. In this work, we provide a systematic evaluation and comparison of three different classes of MARL algorithms (independent learning, centralised multi-agent policy gradient, value decomposition) in a diverse range of cooperative multi-agent learning tasks. Our experiments serve as a reference for the expected performance of algorithms across different learning tasks, and we provide insights regarding the effectiveness of different learning approaches. We open-source EPyMARL, which extends the PyMARL codebase to include additional algorithms and allow for flexible configuration of algorithm implementation details such as parameter sharing. Finally, we open-source two environments for multi-agent research which focus on coordination under sparse rewards.

Cited by

• StarCraft+: Benchmarking Multi-agent Algorithms in Adversary Paradigm
• MARL Warehouse Robots
• MAESTRO: Multi-Agent Environment Shaping through Task and Reward Optimization
• Think How Your Teammates Think: Active Inference Can Benefit Decentralized Execution
• AdaFair-MARL: Enforcing Adaptive Fairness Constraints in Multi-Agent Reinforcement Learning
• Graph Neural Networks, Deep Reinforcement Learning and Probabilistic Topic Modeling for Strategic Multiagent Settings
• Explaining Decentralized Multi-Agent Reinforcement Learning Policies
• Scaling Multi-Agent Environment Co-Design with Diffusion Models
• Multi-party Agent Relation Sampling for Multi-party Ad Hoc Teamwork
• Inter-Agent Relative Representations for Multi-Agent Option Discovery
• High-order Interactions Modeling for Interpretable Multi-Agent Q-Learning
• Diffusion Models for Reinforcement Learning: Foundations, Taxonomy, and Development
• Empirical Study on Robustness and Resilience in Cooperative Multi-Agent Reinforcement Learning
• Climate Surrogates for Scalable Multi-Agent Reinforcement Learning: A Case Study with CICERO-SCM
• Wonder Wins Ways: Curiosity-Driven Exploration through Multi-Agent Contextual Calibration
• Adaptive Event-Triggered Policy Gradient for Multi-Agent Reinforcement Learning
• Learning to Lead Themselves: Agentic AI in MAS using MARL
• The Heterogeneous Multi-Agent Challenge
• MARS-RA: Rank Aggregation for Credit Assignment via Multimodal Comparisons in Embodied Multi-Agent Cooperation
• Pareto Actor-Critic for Communication and Computation Co-Optimization in Non-Cooperative Federated Learning Services
• When Is Diversity Rewarded in Cooperative Multi-Agent Learning?
• cMALC-D: Contextual Multi-Agent LLM-Guided Curriculum Learning with Diversity-Based Context Blending
• Skill-Aligned Fairness in Multi-Agent Learning for Collaboration in Healthcare
• MACTAS: Self-Attention-Based Inter-Agent Communication in Multi-Agent Reinforcement Learning with Action-Value Function Decomposition
• CAMAR: Continuous Actions Multi-Agent Routing
• Constrained Black-Box Attacks Against Multi-Agent Reinforcement Learning
• Multi-level Advantage Credit Assignment for Cooperative Multi-Agent Reinforcement Learning
• Unsupervised Partner Design Enables Robust Ad-hoc Teamwork
• Centralized Adaptive Sampling for Reliable Co-Training of Independent Multi-Agent Policies
• Hierarchical Message-Passing Policies for Multi-Agent Reinforcement Learning
• Enhancing Multi-Agent Collaboration with Attention-Based Actor-Critic Policies
• "Teammates, Am I Clear?": Analysing Legible Behaviours in Teams
• Moving Out: Physically-grounded Human-AI Collaboration
• Multi-Agent Guided Policy Optimization
• Remembering the Markov Property in Cooperative MARL
• From General Relation Patterns to Task-Specific Decision-Making in Continual Multi-Agent Coordination
• MARBLE: A Multi-Agent Rule-Based LLM Reasoning Engine for Accident Severity Prediction
• Curriculum Learning With Counterfactual Group Relative Policy Advantage For Multi-Agent Reinforcement Learning
• Transformer World Model for Sample Efficient Multi-Agent Reinforcement Learning
• Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunities
• Rainbow Delay Compensation: A Multi-Agent Reinforcement Learning Framework for Mitigating Delayed Observation
• Action Dependency Graphs for Globally Optimal Coordinated Reinforcement Learning
• Oryx: a Scalable Sequence Model for Many-Agent Coordination in Offline MARL
• Topology-Aware and Highly Generalizable Deep Reinforcement Learning for Efficient Retrieval in Multi-Deep Storage Systems
• Integrating Counterfactual Simulations with Language Models for Explaining Multi-Agent Behaviour
• URB -- Urban Routing Benchmark for RL-equipped Connected Autonomous Vehicles
• Coordinated Spatial Reuse Scheduling With Machine Learning in IEEE 802.11 MAPC Networks
• JaxRobotarium: Training and Deploying Multi-Robot Policies in 10 Minutes
• Enhancing Cooperative Multi-Agent Reinforcement Learning with State Modelling and Adversarial Exploration
• Safe and Policy-Compliant Multi-Agent Orchestration for Enterprise AI
• QLLM: Do We Really Need a Mixing Network for Credit Assignment in Multi-Agent Reinforcement Learning?
• SDHN: Skewness-Driven Hypergraph Networks for Enhanced Localized Multi-Robot Coordination

Related

• Towards a Standardised Performance Evaluation Protocol for Cooperative MARL
• An Extended Benchmarking of Multi-Agent Reinforcement Learning Algorithms in Complex Fully Cooperative Tasks
• A Review of Cooperative Multi-Agent Deep Reinforcement Learning
• Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learning
• An Initial Introduction to Cooperative Multi-Agent Reinforcement Learning