Model-Based Reinforcement Learning for Approximate Optimal Control with\n Temporal Logic Specifications

2021/01/18 by Max M. Cohen, Cohen, Max, Călin Belta +1 · 1 citation
Biochemistry, Genetics and Molecular Biology · Engineering · #Receptor Mechanisms and Signaling #Advanced Control Systems Optimization

paper · pdf · doi:10.48550/arxiv.2101.07156

Abstract

In this paper we study the problem of synthesizing optimal control policies\nfor uncertain continuous-time nonlinear systems from syntactically co-safe\nlinear temporal logic (scLTL) formulas. We formulate this problem as a sequence\nof reach-avoid optimal control sub-problems. We show that the resulting hybrid\noptimal control policy guarantees the satisfaction of a given scLTL formula by\nconstructing a barrier certificate. Since solving each optimal control problem\nmay be computationally intractable, we take a learning-based approach to\napproximately solve this sequence of optimal control problems online without\nrequiring full knowledge of the system dynamics. Using Lyapunov-based tools, we\ndevelop sufficient conditions under which our approximate solution maintains\ncorrectness. Finally, we demonstrate the efficacy of the developed method with\na numerical example.\n

Cited by

• About Time: Model-free Reinforcement Learning with Timed Reward Machines

Related

• Safe reinforcement learning control for continuous-time nonlinear systems without a backup controller
• Safe Model-Based Reinforcement Learning for Systems with Parametric Uncertainties
• Control Synthesis from Linear Temporal Logic Specifications using\n Model-Free Reinforcement Learning
• Regret-Free Reinforcement Learning for LTL Specifications
• Certified Reinforcement Learning with Logic Guidance