Safe Deep Model-Based Reinforcement Learning with Lyapunov Functions

2024/05/25 by Harry Zhang, Zhang, Harry · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Reinforcement Learning in Robotics #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2405.16184

openalex publication_date 2024/05/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Model-based Reinforcement Learning (MBRL) has shown many desirable properties for intelligent control tasks. However, satisfying safety and stability constraints during training and rollout remains an open question. We propose a new Model-based RL framework to enable efficient policy learning with unknown dynamics based on learning model predictive control (LMPC) framework with mathematically provable guarantees of stability. We introduce and explore a novel method for adding safety constraints for model-based RL during training and policy learning. The new stability-augmented framework consists of a neural-network-based learner that learns to construct a Lyapunov function, and a model-based RL agent to consistently complete the tasks while satisfying user-specified constraints given only sub-optimal demonstrations and sparse-cost feedback. We demonstrate the capability of the proposed framework through simulated experiments.

Cited by

• H2OFlow: Grounding Human-Object Affordances with 3D Generative Models and Dense Diffused Flows

Related

• Safe Model-based Reinforcement Learning with Stability Guarantees
• A Review On Safe Reinforcement Learning Using Lyapunov and Barrier Functions
• Certifying Stability of Reinforcement Learning Policies using Generalized Lyapunov Functions
• Neural Lyapunov Model Predictive Control: Learning Safe Global Controllers from Sub-optimal Examples
• Stable and Safe Reinforcement Learning via a Barrier-Lyapunov Actor-Critic Approach