Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Control

2019/09/05 by Sai Qian Zhang, Qi Zhang, Zhang, Sai Qian +3 · 8 citations
Computer Science · Mathematics · #Adaptive Dynamic Programming Control #Distributed Control Multi-Agent Systems #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1909.02682

openalex publication_date 2019/09/06 · arxiv created 2019/11/01 · arxiv updated 2019/11/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Multi-agent reinforcement learning (MARL) has recently received considerable attention due to its applicability to a wide range of real-world applications. However, achieving efficient communication among agents has always been an overarching problem in MARL. In this work, we propose Variance Based Control (VBC), a simple yet efficient technique to improve communication efficiency in MARL. By limiting the variance of the exchanged messages between agents during the training phase, the noisy component in the messages can be eliminated effectively, while the useful part can be preserved and utilized by the agents for better performance. Our evaluation using a challenging set of StarCraft II benchmarks indicates that our method achieves 2-10× lower in communication overhead than state-of-the-art MARL algorithms, while allowing agents to better collaborate by developing sophisticated strategies.

Citations

• Playing Atari with Deep Reinforcement Learning
• Reinforcement learning in robotics: A survey
• Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving
• Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games
• Learning to Schedule Communication in Multi-agent Reinforcement Learning
• Counterfactual Multi-Agent Policy Gradients

Cited by

• Multi-Agent Reinforcement Learning with Communication-Constrained Priors
• VIL2C: Value-of-Information Aware Low-Latency Communication for Multi-Agent Reinforcement Learning
• Goal-Oriented Multi-Agent Reinforcement Learning for Decentralized Agent Teams
• Learning Communication Skills in Multi-task Multi-agent Deep Reinforcement Learning
• Learning To Communicate Over An Unknown Shared Network
• Towards Language-Augmented Multi-Agent Deep Reinforcement Learning
• Robust Multi-agent Communication Based on Decentralization-Oriented Adversarial Training
• Multi-Agent Reinforcement Learning for Resources Allocation Optimization: A Survey

Related

• Taming Multi-Agent Reinforcement Learning with Estimator Variance Reduction
• Reducing Variance Caused by Communication in Decentralized Multi-agent Deep Reinforcement Learning
• Succinct and Robust Multi-Agent Communication With Temporal Message Control
• Coordinating Policies Among Multiple Agents via an Intelligent Communication Channel
• Effective Communications: A Joint Learning and Communication Framework for Multi-Agent Reinforcement Learning over Noisy Channels