Approximate Dynamic Programming For Linear Systems with State and Input Constraints

2019/06/26 by Ankush Chakrabarty, Chakrabarty, Ankush, Rien Quirynen +5
Computer Science · Engineering · #Adaptive Dynamic Programming Control #Dynamical Systems (math.DS) #FOS: Computer and information sciences #FOS: Electrical engineering #FOS: Mathematics #Machine Learning (cs.LG) #Mechanical Circulatory Support Devices #Optimization and Control (math.OC) #Reinforcement Learning in Robotics #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.1906.11369

openalex publication_date 2019/06/26 · openalex created_date 2019/07/12 · openalex updated_date 2026/07/28

Abstract

Enforcing state and input constraints during reinforcement learning (RL) in continuous state spaces is an open but crucial problem which remains a roadblock to using RL in safety-critical applications. This paper leverages invariant sets to update control policies within an approximate dynamic programming (ADP) framework that guarantees constraint satisfaction for all time and converges to the optimal policy (in a linear quadratic regulator sense) asymptotically. An algorithm for implementing the proposed constrained ADP approach in a data-driven manner is provided. The potential of this formalism is demonstrated via numerical examples.

Citations

• Semidefinite Programming

Related

• Safe Approximate Dynamic Programming Via Kernelized Lipschitz Estimation
• Deterministic Policies for Constrained Reinforcement Learning in Polynomial Time
• A Supplementary Condition for the Convergence of the Control Policy during Adaptive Dynamic Programming
• Safe Model-Based Reinforcement Learning for Systems with Parametric Uncertainties
• State Augmented Constrained Reinforcement Learning: Overcoming the Limitations of Learning with Rewards