Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games

2017/03/29 by Peng Peng, Peng, Peng, Ying Wen +11 · 2 voices · 19 citations
Computer Science · #Adversarial Robustness in Machine Learning #Artificial Intelligence in Games #Reinforcement Learning in Robotics #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.1703.10069

openalex publication_date 2017/03/29 · openalex created_date 2017/09/25 · openalex updated_date 2026/07/28

Abstract

Many artificial intelligence (AI) applications often require multiple intelligent agents to work in a collaborative effort. Efficient learning for intra-agent communication and coordination is an indispensable step towards general AI. In this paper, we take StarCraft combat game as a case study, where the task is to coordinate multiple agents as a team to defeat their enemies. To maintain a scalable yet effective communication protocol, we introduce a Multiagent Bidirectionally-Coordinated Network (BiCNet ['bIknet]) with a vectorised extension of actor-critic formulation. We show that BiCNet can handle different types of combats with arbitrary numbers of AI agents for both sides. Our analysis demonstrates that without any supervisions such as human demonstrations or labelled data, BiCNet could learn various types of advanced coordination strategies that have been commonly used by experienced game players. In our experiments, we evaluate our approach against multiple baselines under different scenarios; it shows state-of-the-art performance, and possesses potential values for large-scale real-world applications.

Citations

• Deep learning in neural networks: An overview
• Bidirectional recurrent neural networks
• Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups
• Episodic Exploration for Deep Deterministic Policies: An Application to\n StarCraft Micromanagement Tasks
• TorchCraft: a Library for Machine Learning Research on Real-Time Strategy Games

Cited by

• VIL2C: Value-of-Information Aware Low-Latency Communication for Multi-Agent Reinforcement Learning
• Learning what to say and how precisely: Efficient Communication via Differentiable Discrete Communication Learning
• When2com: Multi-Agent Perception via Communication Graph Grouping
• StarCraft II: A New Challenge for Reinforcement Learning
• Learning Nearly Decomposable Value Functions Via Communication Minimization
• Value-Decomposition Multi-Agent Actor-Critics
• Multi-agent Deep Reinforcement Learning with Extremely Noisy\n Observations
• Efficient Ridesharing Order Dispatching with Mean Field Multi-Agent Reinforcement Learning
• STARDATA: A StarCraft AI Research Dataset
• HLSMAC: A New StarCraft Multi-Agent Challenge for High-Level Strategic Decision-Making
• Who2com: Collaborative Perception via Learnable Handshake Communication
• A Comprehensive Review of Multi-Agent Reinforcement Learning in Video Games
• Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent\n Learning Systems
• Mean Field Multi-Agent Reinforcement Learning
• MSC: A Dataset for Macro-Management in StarCraft II
• KnowSR: Knowledge Sharing among Homogeneous Agents in Multi-agent Reinforcement Learning
• Efficient Ridesharing Dispatch Using Multi-Agent Reinforcement Learning
• Adaptive Command: Real-Time Policy Adjustment via Language Models in StarCraft II
• Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning

Discussions

• Multiagent Bidirectionally-Coordinated Nets for Learning to Play StarCraft [hn, 4 points, 0 comments]
• Multiagent Bidirectionally-Coordinated Nets for Playing StarCraft [hn, 3 points, 0 comments]

Related

• Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks
• Multiplayer Games for Learning Multirobot Coordination Algorithms
• Action Semantics Network: Considering the Effects of Actions in Multiagent Systems
• FCMNet: Full Communication Memory Net for Team-Level Cooperation in Multi-Agent Systems
• Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination