Reinforcement Learning for Safety-Critical Control under Model\n Uncertainty, using Control Lyapunov Functions and Control Barrier Functions

2020/04/16 by Jason Choi, Fernando Castañeda, Choi, Jason +5 · 8 citations
Computer Science · Engineering · #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Prosthetics and Rehabilitation Robotics #Reinforcement Learning in Robotics #Robotic Locomotion and Control #Robotics (cs.RO) #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2004.07584

openalex publication_date 2020/04/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, the issue of model uncertainty in safety-critical control is\naddressed with a data-driven approach. For this purpose, we utilize the\nstructure of an input-ouput linearization controller based on a nominal model\nalong with a Control Barrier Function and Control Lyapunov Function based\nQuadratic Program (CBF-CLF-QP). Specifically, we propose a novel reinforcement\nlearning framework which learns the model uncertainty present in the CBF and\nCLF constraints, as well as other control-affine dynamic constraints in the\nquadratic program. The trained policy is combined with the nominal model-based\nCBF-CLF-QP, resulting in the Reinforcement Learning-based CBF-CLF-QP\n(RL-CBF-CLF-QP), which addresses the problem of model uncertainty in the safety\nconstraints. The performance of the proposed method is validated by testing it\non an underactuated nonlinear bipedal robot walking on randomly spaced stepping\nstones with one step preview, obtaining stable and safe walking under model\nuncertainty.\n

Cited by

• Online Learning-Enhanced High Order Adaptive Safety Control
• Robust Time-Varying Control Barrier Functions with Sector-Bounded Nonlinearities
• Survey and Tutorial of Reinforcement Learning Methods in Process Systems Engineering
• EigenSafe: A Spectral Framework for Learning-Based Probabilistic Safety Assessment
• CLF-RL: Control Lyapunov Function Guided Reinforcement Learning
• A Review On Safe Reinforcement Learning Using Lyapunov and Barrier Functions
• End-to-End Humanoid Robot Safe and Comfortable Locomotion Policy
• Invariance Guarantees using Continuously Parametrized Control Barrier Functions

Related

• Reinforcement Learning for Safe Robot Control using Control Lyapunov Barrier Functions
• Robust Safety-Critical Control for Dynamic Robotics
• Certifiable Safe Model-Based Reinforcement Learning with Control-Affine Dynamics Approximation
• Bridging Model-based Safety and Model-free Reinforcement Learning through System Identification of Low Dimensional Linear Models
• A Review On Safe Reinforcement Learning Using Lyapunov and Barrier Functions