Learning Efficient Multi-agent Communication: An Information Bottleneck Approach

2019/11/16 by Rundong Wang, Xu He, Wang, Rundong +9 · 5 citations
Computer Science · Engineering · #Advanced Memory and Neural Computing #Artificial Intelligence (cs.AI) #Distributed Control Multi-Agent Systems #FOS: Computer and information sciences #Multiagent Systems (cs.MA) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.1911.06992

openalex publication_date 2019/11/16 · openalex created_date 2019/11/22 · openalex updated_date 2026/07/28

Abstract

We consider the problem of the limited-bandwidth communication for multi-agent reinforcement learning, where agents cooperate with the assistance of a communication protocol and a scheduler. The protocol and scheduler jointly determine which agent is communicating what message and to whom. Under the limited bandwidth constraint, a communication protocol is required to generate informative messages. Meanwhile, an unnecessary communication connection should not be established because it occupies limited resources in vain. In this paper, we develop an Informative Multi-Agent Communication (IMAC) method to learn efficient communication protocols as well as scheduling. First, from the perspective of communication theory, we prove that the limited bandwidth constraint requires low-entropy messages throughout the transmission. Then inspired by the information bottleneck principle, we learn a valuable and compact communication protocol and a weight-based scheduler. To demonstrate the efficiency of our method, we conduct extensive experiments in various cooperative and competitive multi-agent tasks with different numbers of agents and different bandwidths. We show that IMAC converges faster and leads to efficient communication among agents under the limited bandwidth as compared to many baseline methods.

Citations

• The information bottleneck method
• Multi-agent Deep Reinforcement Learning with Extremely Noisy\n Observations
• LEARNING TO SCHEDULE COMMUNICATION IN MULTI-AGENT REINFORCEMENT LEARNING

Cited by

• Single-Agent Scaling Fails Multi-Agent Intelligence: Towards Foundation Models with Native Multi-Agent Intelligence
• VIL2C: Value-of-Information Aware Low-Latency Communication for Multi-Agent Reinforcement Learning
• Learning what to say and how precisely: Efficient Communication via Differentiable Discrete Communication Learning
• Towards Minimal Causal Representations for Human Multimodal Language Understanding
• Toward Goal-Oriented Communication in Multi-Agent Systems: An overview

Related

• Event-Triggered Multi-agent Reinforcement Learning with Communication under Limited-bandwidth Constraint
• Learning Efficient Communication Protocols for Multi-Agent Reinforcement Learning
• Minimizing Communication while Maximizing Performance in Multi-Agent Reinforcement Learning
• LEARNING TO SCHEDULE COMMUNICATION IN MULTI-AGENT REINFORCEMENT LEARNING
• Learning Agent Communication under Limited Bandwidth by Message Pruning