Safe model-based Reinforcement Learning via Model Predictive Control and Control Barrier Functions

2025/12/04 by Dzhumageldyev, Kerim, Airaldi, Filippo, Dabiri, Azita
#FOS: Electrical engineering #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2512.04856

Abstract

Optimal control strategies are often combined with safety certificates to ensure both performance and safety in safety-critical systems. A prominent example is combining Model Predictive Control (MPC) with Control Barrier Functions (CBF). Yet, efficient tuning of MPC parameters and choosing an appropriate class K function in the CBF is challenging and problem dependent. This paper introduces a safe model-based Reinforcement Learning (RL) framework where a parametric MPC controller incorporates a CBF constraint with a parameterized class K function and serves as a function approximator to learn improved safe control policies from data. Three variations of the framework are introduced, distinguished by the way the optimization problem is formulated and the class K function is parameterized, including neural architectures. Numerical experiments on a discrete double-integrator with static and dynamic obstacles demonstrate that the proposed methods improve performance while ensuring safety.

Citations

• When Do Transformers Outperform Feedforward and Recurrent Networks? A Statistical Perspective
• Reinforcement Learning with Model Predictive Control for Highway Ramp Metering
• On the Optimality, Stability, and Feasibility of Control Barrier Functions: An Adaptive Learning-Based Approach
• Enhancing Feasibility and Safety of Nonlinear Model Predictive Control with Discrete-Time Control Barrier Functions
• Safety-Critical Control using Optimal-decay Control Barrier Function with Guaranteed Point-wise Feasibility
• Safety-Critical Model Predictive Control with Discrete-Time Control Barrier Function
• Control Barrier Functions: Theory and Applications
• Adam: A Method for Stochastic Optimization
• Finding Structure in Time

Related

• Safe Model-Based Reinforcement Learning for Systems with Parametric Uncertainties
• Probabilistically safe and efficient model-based reinforcement learning
• Safe Reinforcement Learning with Chance-constrained Model Predictive Control
• Safety aware model-based reinforcement learning for optimal control of a class of output-feedback nonlinear systems
• Learning for Safety-Critical Control with Control Barrier Functions