Certified Reinforcement Learning with Logic Guidance

2019/02/02 by Hasanbeig, Hosein, Kroening, Daniel, Abate, Alessandro · 2 citations
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.1902.00778

Abstract

Reinforcement Learning (RL) is a widely employed machine learning architecture that has been applied to a variety of control problems. However, applications in safety-critical domains require a systematic and formal approach to specifying requirements as tasks or goals. We propose a model-free RL algorithm that enables the use of Linear Temporal Logic (LTL) to formulate a goal for unknown continuous-state/action Markov Decision Processes (MDPs). The given LTL property is translated into a Limit-Deterministic Generalised Buchi Automaton (LDGBA), which is then used to shape a synchronous reward function on-the-fly. Under certain assumptions, the algorithm is guaranteed to synthesise a control policy whose traces satisfy the LTL specification with maximal probability.

Cited by

• Reinforcement Learning with ω-Regular Objectives and Constraints
• Physics-Informed Reward Machines

Related

• Reinforcement Learning for Temporal Logic Control Synthesis with\n Probabilistic Satisfaction Guarantees
• Reinforcement Learning Based Temporal Logic Control with Soft Constraints Using Limit-deterministic Generalized Buchi Automata
• Logically-Constrained Reinforcement Learning
• Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learning
• LCRL: Certified Policy Synthesis via Logically-Constrained Reinforcement Learning