A Reinforcement Learning Approach for Intelligent Traffic Signal Control at Urban Intersections

2019/05/19 by Mengyu Guo, Pin Wang, Guo, Mengyu +5 · 1 citation
Engineering · Social Sciences · #FOS: Electrical engineering #Signal Processing (eess.SP) #Systems and Control (eess.SY) #Traffic Prediction and Management Techniques #Traffic control and management #Transportation Planning and Optimization #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.1905.07698

openalex publication_date 2019/05/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Ineffective and inflexible traffic signal control at urban intersections can often lead to bottlenecks in traffic flows and cause congestion, delay, and environmental problems. How to manage traffic smartly by intelligent signal control is a significant challenge in urban traffic management. With recent advances in machine learning, especially reinforcement learning (RL), traffic signal control using advanced machine learning techniques represents a promising solution to tackle this problem. In this paper, we propose a RL approach for traffic signal control at urban intersections. Specifically, we use neural networks as Q-function approximator (a.k.a. Q-network) to deal with the complex traffic signal control problem where the state space is large and the action space can be discrete. The state space is defined based on real-time traffic information, i.e. vehicle position, direction and speed. The action space includes various traffic signal phases which are critical in generating a reasonable and realistic control mechanism, given the prominent spatial-temporal characteristics of urban traffic. In the simulation experiment, we use SUMO, an open source traffic simulator, to construct realistic urban intersection settings. Moreover, we use different traffic patterns, such as major/minor road traffic, through/left-turn lane traffic, tidal traffic, and varying demand traffic, to train a generalized traffic signal control model that can be adapted to various traffic conditions. The simulation results demonstrate the convergence and generalization performance of our RL approach as well as its significant benefits in terms of queue length and wait time over several benchmarking methods in traffic signal control.

Citations

• Dueling Network Architectures for Deep Reinforcement Learning

Cited by

• CFLight: Enhancing Safety with Traffic Signal Control through Counterfactual Learning

Related

• Reinforcement Learning for Adaptive Traffic Signal Control: Turn-Based and Time-Based Approaches to Reduce Congestion
• Reinforcement Learning Based Traffic Signal Design to Minimize Queue Lengths
• Adaptive Traffic Signal Control: Deep Reinforcement Learning Algorithm with Experience Replay and Target Network
• A Survey on Traffic Signal Control Methods
• Using a Deep Reinforcement Learning Agent for Traffic Signal Control