2021/10/15 by Siyang Wu, Tonghan Wang, Wu, Siyang +5
Computer Science · #Data Stream Mining Techniques #FOS: Computer and information sciences #Machine Learning (cs.LG) #Mobile Crowdsensing and Crowdsourcing #Reinforcement Learning in Robotics
paper · pdf · doi:10.48550/arxiv.2110.08169
openalex publication_date 2021/10/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Multi-agent reinforcement learning tasks put a high demand on the volume of training samples. Different from its single-agent counterpart, distributed value-based multi-agent reinforcement learning faces the unique challenges of demanding data transfer, inter-process communication management, and high requirement of exploration. We propose a containerized learning framework to solve these problems. We pack several environment instances, a local learner and buffer, and a carefully designed multi-queue manager which avoids blocking into a container. Local policies of each container are encouraged to be as diverse as possible, and only trajectories with highest priority are sent to a global learner. In this way, we achieve a scalable, time-efficient, and diverse distributed MARL learning framework with high system throughput. To own knowledge, our method is the first to solve the challenging Google Research Football full game 5_v_5. On the StarCraft II micromanagement benchmark, our method gets 4-18× better results compared to state-of-the-art non-distributed MARL algorithms.