Communicating Plans, Not Percepts: Scalable Multi-Agent Coordination with Embodied World Models

2025/08/04 by Hill, Brennen A., Wei, Mant Koh En, Thangavel Jishnuanandh +1 · 1 citation
Computer Science · #68T05 #68T07 #68T42 #90C40 #93E35 #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #FOS: Electrical engineering #I.2.11 #I.2.6 #I.2.8 #Machine Learning (cs.LG) #Multi-Agent Systems and Negotiation #Multiagent Systems (cs.MA) #Semantic Web and Ontologies #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2508.02912

openalex publication_date 2025/08/04 · openalex created_date 2025/10/19 · openalex updated_date 2026/07/28

Abstract

Robust coordination is critical for effective decision-making in multi-agent systems, especially under partial observability. A central question in Multi-Agent Reinforcement Learning (MARL) is whether to engineer communication protocols or learn them end-to-end. We investigate this dichotomy using embodied world models. We propose and compare two communication strategies for a cooperative task-allocation problem. The first, Learned Direct Communication (LDC), learns a protocol end-to-end. The second, Intention Communication, uses an engineered inductive bias: a compact, learned world model, the Imagined Trajectory Generation Module (ITGM), which uses the agent's own policy to simulate future states. A Message Generation Network (MGN) then compresses this plan into a message. We evaluate these approaches on goal-directed interaction in a grid world, a canonical abstraction for embodied AI problems, while scaling environmental complexity. Our experiments reveal that while emergent communication is viable in simple settings, the engineered, world model-based approach shows superior performance, sample efficiency, and scalability as complexity increases. These findings advocate for integrating structured, predictive models into MARL agents to enable active, goal-driven coordination.

Citations

• Towards Collaborative Intelligence: Propagating Intentions and Reasoning for Multi-Agent Coordination with Large Language Models
• Imagine, Initialize, and Explore: An Effective Exploration Method in Multi-Agent Reinforcement Learning
• Mastering Diverse Domains through World Models
• Multi-Agent Intention Sharing via Leader-Follower Forest
• Mastering Atari with Discrete World Models
• PettingZoo: Gym for Multi-Agent Reinforcement Learning
• Dream to Control: Learning Behaviors by Latent Imagination
• Leveraging Procedural Generation to Benchmark Reinforcement Learning
• On the Utility of Learning about Humans for Human-AI Coordination
• A Review of Cooperative Multi-Agent Deep Reinforcement Learning
• Google Research Football: A Novel Reinforcement Learning Environment
• The StarCraft Multi-Agent Challenge
• Modeling Others using Oneself in Multi-Agent Reinforcement Learning
• Attention Is All You Need
• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
• Emergence of Grounded Compositional Language in Multi-Agent Populations
• Learning Multiagent Communication with Backpropagation
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• The Complexity of Decentralized Control of Markov Decision Processes
• The Complexity of Decentralized Control of Markov Decision Processes
• Reinforcement Learning: An Introduction

Cited by

• Rethinking Multi-Agent Intelligence Through the Lens of Small-World Networks

Related

• Dreamer-CPC: Message Learning with World Models for Decentralized Multi-agent Reinforcement Learning
• HAMMER: Multi-Level Coordination of Reinforcement Learning Agents via Learned Messaging
• Learn as Individuals, Evolve as a Team: Multi-agent LLMs Adaptation in Embodied Environments
• Towards Collaborative Intelligence: Propagating Intentions and Reasoning for Multi-Agent Coordination with Large Language Models
• Ego-centric Learning of Communicative World Models for Autonomous Driving