Incorporating Recurrent Reinforcement Learning into Model Predictive Control for Adaptive Control in Autonomous Driving

2023/01/30 by Yuan Zhang, Zhang, Yuan, Joschka Boedecker +5
Computer Science · Engineering · Medicine · #Artificial Intelligence (cs.AI) #Cardiovascular Function and Risk Factors #FOS: Computer and information sciences #Machine Learning (cs.LG) #Real-time simulation and control systems #Reinforcement Learning in Robotics #Robotics (cs.RO)

paper · pdf · doi:10.48550/arxiv.2301.13313

openalex publication_date 2023/01/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/04

Abstract

Model Predictive Control (MPC) is attracting tremendous attention in the autonomous driving task as a powerful control technique. The success of an MPC controller strongly depends on an accurate internal dynamics model. However, the static parameters, usually learned by system identification, often fail to adapt to both internal and external perturbations in real-world scenarios. In this paper, we firstly (1) reformulate the problem as a Partially Observed Markov Decision Process (POMDP) that absorbs the uncertainties into observations and maintains Markov property into hidden states; and (2) learn a recurrent policy continually adapting the parameters of the dynamics model via Recurrent Reinforcement Learning (RRL) for optimal and adaptive control; and (3) finally evaluate the proposed algorithm (referred as MPC-RRL) in CARLA simulator and leading to robust behaviours under a wide range of perturbations.

Related

• Reinforcement Learning of the Prediction Horizon in Model Predictive\n Control
• Learning Residual Model of Model Predictive Control via Random Forests for Autonomous Driving
• Adaptive Stochastic Nonlinear Model Predictive Control with Look-ahead Deep Reinforcement Learning for Autonomous Vehicle Motion Control
• A Safe Reinforcement Learning driven Weights-varying Model Predictive Control for Autonomous Vehicle Motion Control
• Safety Reinforced Model Predictive Control (SRMPC): Improving MPC with Reinforcement Learning for Motion Planning in Autonomous Driving