Temporal Logic Guided Safe Reinforcement Learning Using Control Barrier Functions

2019/03/23 by Xiao Li, Calin Belta, Li, Xiao +2 · 31 citations
Computer Science · Engineering · Mathematics · #Artificial intelligence #Autonomous Vehicle Technology and Safety #Computer science #Control (management) #FOS: Computer and information sciences #Formal Methods in Verification #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics #Reinforcement learning #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1903.09885

published in arXiv (Cornell University) (Cornell University)

arxiv created 2019/03/23 · openalex publication_date 2019/03/23 · arxiv updated 2019/03/26 · openalex created_date 2019/04/01 · openalex updated_date 2026/07/28

Abstract

Using reinforcement learning to learn control policies is a challenge when the task is complex with potentially long horizons. Ensuring adequate but safe exploration is also crucial for controlling physical systems. In this paper, we use temporal logic to facilitate specification and learning of complex tasks. We combine temporal logic with control Lyapunov functions to improve exploration. We incorporate control barrier functions to safeguard the exploration and deployment process. We develop a flexible and learnable system that allows users to specify task objectives and constraints in different forms and at various levels. The framework is also able to take advantage of known system dynamics and handle unknown environmental dynamics by integrating model-free learning with model-based planning.

Citations

• Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards
• Safe Exploration in Continuous Action Spaces
• The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communities

Cited by

• A Review On Safe Reinforcement Learning Using Lyapunov and Barrier Functions
• Failing with Grace: Learning Neural Network Controllers that are Boundedly Unsafe

Related

• Safety-Critical Learning of Robot Control with Temporal Logic Specifications
• Towards Safe Continuing Task Reinforcement Learning
• Prescribed Performance Control Guided Policy Improvement for Satisfying Signal Temporal Logic Tasks
• A Policy Search Method For Temporal Logic Specified Reinforcement Learning Tasks
• Sample-Efficient Reinforcement Learning with Temporal Logic Objectives: Leveraging the Task Specification to Guide Exploration