Action Semantics Network: Considering the Effects of Actions in Multiagent Systems

2019/07/26 by Weixun Wang, Tianpei Yang, Wang, Weixun +15 · 3 citations
Computer Science · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Games #Evolutionary Algorithms and Applications #FOS: Computer and information sciences #Multiagent Systems (cs.MA) #Reinforcement Learning in Robotics #cs.AI #cs.MA

paper · pdf · doi:10.48550/arxiv.1907.11461

accepted by ICLR2020

openalex publication_date 2019/07/26 · arxiv created 2020/01/16 · arxiv updated 2020/01/17 · openalex created_date 2022/07/28 · openalex updated_date 2026/07/28

Abstract

In multiagent systems (MASs), each agent makes individual decisions but all of them contribute globally to the system evolution. Learning in MASs is difficult since each agent's selection of actions must take place in the presence of other co-learning agents. Moreover, the environmental stochasticity and uncertainties increase exponentially with the increase in the number of agents. Previous works borrow various multiagent coordination mechanisms into deep learning architecture to facilitate multiagent coordination. However, none of them explicitly consider action semantics between agents that different actions have different influences on other agents. In this paper, we propose a novel network architecture, named Action Semantics Network (ASN), that explicitly represents such action semantics between agents. ASN characterizes different actions' influence on other agents using neural networks based on the action semantics between them. ASN can be easily combined with existing deep reinforcement learning (DRL) algorithms to boost their performance. Experimental results on StarCraft II micromanagement and Neural MMO show ASN significantly improves the performance of state-of-the-art DRL approaches compared with several network architectures.

Citations

• Dueling Network Architectures for Deep Reinforcement Learning
• Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
• Relational Forward Models for Multi-Agent Learning
• Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents
• Counterfactual Multi-Agent Policy Gradients

Cited by

• IFS: Information Flow Structure for Multi-agent Ad Hoc System
• High-order Interactions Modeling for Interpretable Multi-Agent Q-Learning
• Fine-Grained AI Model Caching and Downloading With Coordinated Multipoint Broadcasting in Multi-Cell Edge Networks

Related

• Revisiting the Master-Slave Architecture in Multi-Agent Deep Reinforcement Learning
• Multi-Task Multi-Agent Shared Layers are Universal Cognition of Multi-Agent Coordination
• Deep Implicit Coordination Graphs for Multi-agent Reinforcement Learning
• Attentional Policies for Cross-Context Multi-Agent Reinforcement Learning
• Strategy-Following Multi-Agent Deep Reinforcement Learning Considering Control Strategies Provided to Other Agents