Multi-agent Reinforcement Learning for Networked System Control

2020/04/03 by Tianshu Chu, Chu, Tianshu, Sandeep Chinchali +3 · 11 citations
Computer Science · Engineering · Mathematics · #Reinforcement Learning in Robotics #Smart Grid Security and Resilience #Traffic control and management #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2004.01339

ICLR 2020

arxiv created 2020/04/24 · arxiv updated 2020/04/27

Abstract

This paper considers multi-agent reinforcement learning (MARL) in networked system control. Specifically, each agent learns a decentralized control policy based on local observations and messages from connected neighbors. We formulate such a networked MARL (NMARL) problem as a spatiotemporal Markov decision process and introduce a spatial discount factor to stabilize the training of each local agent. Further, we propose a new differentiable communication protocol, called NeurComm, to reduce information loss and non-stationarity in NMARL. Based on experiments in realistic NMARL scenarios of adaptive traffic signal control and cooperative adaptive cruise control, an appropriate spatial discount factor effectively enhances the learning curves of non-communicative MARL algorithms, while NeurComm outperforms existing communication protocols in both learning efficiency and control performance.

Cited by

• Graph Convolutional Value Decomposition in Multi-Agent Reinforcement Learning
• Empirical Study on Robustness and Resilience in Cooperative Multi-Agent Reinforcement Learning
• Bayesian Ego-graph Inference for Networked Multi-Agent Reinforcement Learning
• DCT-MARL: A Dynamic Communication Topology-Based MARL Algorithm for Connected Vehicle Platoon Control
• Independent Reinforcement Learning for Weakly Cooperative Multiagent Traffic Control Problem
• Multi-Agent Intention Sharing via Leader-Follower Forest
• Action Dependency Graphs for Globally Optimal Coordinated Reinforcement Learning
• A Decentralized Policy Gradient Approach to Multi-task Reinforcement Learning
• Community-based Multi-Agent Reinforcement Learning with Transfer and Active Exploration
• Multi-agent Embodied AI: Advances and Future Directions
• OnSlicing

Related

• Decentralized Multi-Agent Reinforcement Learning with Networked Agents: Recent Advances
• Toward Multi-Agent Reinforcement Learning for Distributed Event-Triggered Control
• Networked Multi-Agent Reinforcement Learning with Emergent Communication
• Value Propagation for Decentralized Networked Deep Multi-agent Reinforcement Learning
• Scalable and Sample Efficient Distributed Policy Gradient Algorithms in Multi-Agent Networked Systems