Learning Agent Communication under Limited Bandwidth by Message Pruning

2019/12/03 by Hangyu Mao, Mao, Hangyu, Zhengchao Zhang +8 · 10 citations
Computer Science · Engineering · #Artificial Intelligence (cs.AI) #Distributed Control Multi-Agent Systems #FOS: Computer and information sciences #Machine Learning (cs.LG) #Modular Robots and Swarm Intelligence #Multiagent Systems (cs.MA) #Reinforcement Learning in Robotics #cs.AI #cs.LG #cs.MA

paper · pdf · doi:10.48550/arxiv.1912.05304

accepted as a regular paper with poster presentation @ AAAI20. arXiv admin note: text overlap with arXiv:1903.05561

arxiv created 2019/12/03 · openalex publication_date 2019/12/03 · arxiv updated 2019/12/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Communication is a crucial factor for the big multi-agent world to stay organized and productive. Recently, Deep Reinforcement Learning (DRL) has been applied to learn the communication strategy and the control policy for multiple agents. However, the practical \emphlimited bandwidth in multi-agent communication has been largely ignored by the existing DRL methods. Specifically, many methods keep sending messages incessantly, which consumes too much bandwidth. As a result, they are inapplicable to multi-agent systems with limited bandwidth. To handle this problem, we propose a gating mechanism to adaptively prune less beneficial messages. We evaluate the gating mechanism on several tasks. Experiments demonstrate that it can prune a lot of messages with little impact on performance. In fact, the performance may be greatly improved by pruning redundant messages. Moreover, the proposed gating mechanism is applicable to several previous methods, equipping them the ability to address bandwidth restricted settings.

Citations

• Winner-Take-All Autoencoders
• Continuous control with deep reinforcement learning
• Binarized Neural Networks
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• Learning Multiagent Communication with Backpropagation
• A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity
• Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games
• Multi-agent Deep Reinforcement Learning with Extremely Noisy Observations
• Modelling the Dynamic Joint Policy of Teammates with Attention\n Multi-agent DDPG
• Learning Multi-agent Communication under Limited-bandwidth Restriction for Internet Packet Routing

Cited by

• Bandwidth-constrained Variational Message Encoding for Cooperative Multi-agent Reinforcement Learning
• Multi-Agent Reinforcement Learning with Communication-Constrained Priors
• VIL2C: Value-of-Information Aware Low-Latency Communication for Multi-Agent Reinforcement Learning
• Learning Communication Skills in Multi-task Multi-agent Deep Reinforcement Learning
• DCT-MARL: A Dynamic Communication Topology-Based MARL Algorithm for Connected Vehicle Platoon Control
• Learning To Communicate Over An Unknown Shared Network
• Internet of Agents: Fundamentals, Applications, and Challenges
• Learning Value of Information towards Joint Communication and Control in 6G V2X
• NGENT: Next-Generation AI Agents Must Integrate Multi-Domain Abilities to Achieve Artificial General Intelligence
• Robust Multi-agent Communication Based on Decentralization-Oriented Adversarial Training

Related

• Learning Multi-agent Communication under Limited-bandwidth Restriction for Internet Packet Routing
• Minimizing Communication while Maximizing Performance in Multi-Agent Reinforcement Learning
• Learning Efficient Multi-agent Communication: An Information Bottleneck Approach
• Event-Triggered Multi-agent Reinforcement Learning with Communication under Limited-bandwidth Constraint
• Dynamic Size Message Scheduling for Multi-Agent Communication under Limited Bandwidth