Combining Reinforcement Learning with Model Predictive Control for On-Ramp Merging

2020/11/17 by Joseph Lubars, Lubars, Joseph, Harsh Gupta +11 · 2 citations
Computer Science · Engineering · #Artificial Intelligence (cs.AI) #Autonomous Vehicle Technology and Safety #FOS: Computer and information sciences #Machine Learning (cs.LG) #Robotics (cs.RO) #Traffic control and management #Vehicle Dynamics and Control Systems #cs.AI #cs.LG #cs.RO

paper · pdf · doi:10.48550/arxiv.2011.08484

8 pages, 6 figures

openalex publication_date 2020/11/17 · arxiv created 2021/09/28 · arxiv updated 2021/09/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We consider the problem of designing an algorithm to allow a car to autonomously merge on to a highway from an on-ramp. Two broad classes of techniques have been proposed to solve motion planning problems in autonomous driving: Model Predictive Control (MPC) and Reinforcement Learning (RL). In this paper, we first establish the strengths and weaknesses of state-of-the-art MPC and RL-based techniques through simulations. We show that the performance of the RL agent is worse than that of the MPC solution from the perspective of safety and robustness to out-of-distribution traffic patterns, i.e., traffic patterns which were not seen by the RL agent during training. On the other hand, the performance of the RL agent is better than that of the MPC solution when it comes to efficiency and passenger comfort. We subsequently present an algorithm which blends the model-free RL agent with the MPC solution and show that it provides better trade-offs between all metrics -- passenger comfort, efficiency, crash rate and robustness.

Citations

• Playing Atari with Deep Reinforcement Learning
• Congested Traffic States in Empirical Observations and Microscopic Simulations
• Continuous control with deep reinforcement learning

Cited by

• Bootstrapping Reinforcement Learning with Sub-optimal Policies for Autonomous Driving
• A Forward-Inverse Dynamic Game Framework for Enhanced Multi-Agent Trajectory Planning

Related

• Autonomous Highway Merging in Mixed Traffic Using Reinforcement Learning and Motion Predictive Safety Controller
• Safe Reinforcement Learning with Probabilistic Control Barrier Functions for Ramp Merging
• Autonomous Ramp Merge Maneuver Based on Reinforcement Learning with Continuous Action Space
• Reinforcement Learning with Latent State Inference for Autonomous On-ramp Merging under Observation Delay
• Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic