Scalability Bottlenecks in Multi-Agent Reinforcement Learning Systems

2023/02/10 by Kailash Gogineni, Gogineni, Kailash, Wei, Peng +3 · 1 citation
Computer Science · #Data Stream Mining Techniques #Evolutionary Algorithms and Applications #FOS: Computer and information sciences #Multiagent Systems (cs.MA) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2302.05007

openalex publication_date 2023/02/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Multi-Agent Reinforcement Learning (MARL) is a promising area of research that can model and control multiple, autonomous decision-making agents. During online training, MARL algorithms involve performance-intensive computations such as exploration and exploitation phases originating from large observation-action space belonging to multiple agents. In this article, we seek to characterize the scalability bottlenecks in several popular classes of MARL algorithms during their training phases. Our experimental results reveal new insights into the key modules of MARL algorithms that limit the scalability, and outline potential strategies that may help address these performance issues.

Cited by

• Emergence of Roles in Robotic Teams with Model Sharing and Limited Communication

Related

• Towards Efficient Multi-Agent Learning Systems
• Evaluating Generalization and Transfer Capacity of Multi-Agent\n Reinforcement Learning Across Variable Number of Agents
• Scalable Multi-Agent Reinforcement Learning for Networked Systems with Average Reward
• MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Library
• Sable: a Performant, Efficient and Scalable Sequence Model for MARL