STMARL: A Spatio-Temporal Multi-Agent Reinforcement Learning Approach for Cooperative Traffic Light Control

2019/08/28 by Yanan Wang, Tong Xu, Wang, Yanan +9 · 1 citation
Computer Science · Engineering · Social Sciences · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multiagent Systems (cs.MA) #Traffic Prediction and Management Techniques #Traffic control and management #Transportation Planning and Optimization #cs.AI #cs.LG #cs.MA

paper · pdf · doi:10.48550/arxiv.1908.10577

Accepted to IEEE Transactions on Mobile Computing

openalex publication_date 2019/08/28 · openalex created_date 2020/02/24 · arxiv created 2020/11/29 · arxiv updated 2020/12/01 · openalex updated_date 2026/07/28

Abstract

The development of intelligent traffic light control systems is essential for smart transportation management. While some efforts have been made to optimize the use of individual traffic lights in an isolated way, related studies have largely ignored the fact that the use of multi-intersection traffic lights is spatially influenced and there is a temporal dependency of historical traffic status for current traffic light control. To that end, in this paper, we propose a novel SpatioTemporal Multi-Agent Reinforcement Learning (STMARL) framework for effectively capturing the spatio-temporal dependency of multiple related traffic lights and control these traffic lights in a coordinating way. Specifically, we first construct the traffic light adjacency graph based on the spatial structure among traffic lights. Then, historical traffic records will be integrated with current traffic status via Recurrent Neural Network structure. Moreover, based on the temporally-dependent traffic information, we design a Graph Neural Network based model to represent relationships among multiple traffic lights, and the decision for each traffic light will be made in a distributed way by the deep Q-learning method. Finally, the experimental results on both synthetic and real-world data have demonstrated the effectiveness of our STMARL framework, which also provides an insightful understanding of the influence mechanism among multi-intersection traffic lights.

Citations

• Playing Atari with Deep Reinforcement Learning
• Empirically Evaluating Multiagent Learning Algorithms
• Long Short-Term Memory
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• Interaction Networks for Learning about Objects, Relations and Physics
• Neural Message Passing for Quantum Chemistry
• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
• Relational Deep Reinforcement Learning
• Soft Information for Localization-of-Things

Cited by

• Attacking Deep Reinforcement Learning-Based Traffic Signal Control Systems with Colluding Vehicles

Related

• Area-wide traffic signal control based on a deep graph Q-Network (DGQN) trained in an asynchronous manner
• MTLight: Efficient Multi-Task Reinforcement Learning for Traffic Signal Control
• Learning Decentralized Traffic Signal Controllers with Multi-Agent Graph Reinforcement Learning
• Multi-Agent Broad Reinforcement Learning for Intelligent Traffic Light Control
• Optimizing Traffic Lights with Multi-agent Deep Reinforcement Learning and V2X communication