2023/06/06 by Daniel C. H. Tan, Tan, Daniel C. H., Fernando Acero +7 · 1 citation
Computer Science · Engineering · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Formal Methods in Verification #Machine Learning (cs.LG) #Robotics (cs.RO) #Safety Systems Engineering in Autonomy
paper · pdf · doi:10.48550/arxiv.2306.04026
openalex publication_date 2023/06/06 · openalex created_date 2023/06/09 · openalex updated_date 2026/07/28
Guaranteeing safe behaviour of reinforcement learning (RL) policies poses significant challenges for safety-critical applications, despite RL's generality and scalability. To address this, we propose a new approach to apply verification methods from control theory to learned value functions. By analyzing task structures for safety preservation, we formalize original theorems that establish links between value functions and control barrier functions. Further, we propose novel metrics for verifying value functions in safe control tasks and practical implementation details to improve learning. Our work presents a novel method for certificate learning, which unlocks a diversity of verification techniques from control theory for RL policies, and marks a significant step towards a formal framework for the general, scalable, and verifiable design of RL-based control systems. Code and videos are available at this https url: https://rl-cbf.github.io/