2023/04/12 by Nicholas H. Barbara, Ruigang Wang, Barbara, Nicholas H. +3 · 1 citation
Computer Science · Engineering · #Adaptive Dynamic Programming Control #FOS: Computer and information sciences #FOS: Electrical engineering #FOS: Mathematics #Fuel Cells and Related Materials #Machine Learning (cs.LG) #Optimization and Control (math.OC) #Reinforcement Learning in Robotics #Systems and Control (eess.SY) #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2304.06193
openalex publication_date 2023/04/12 · openalex created_date 2023/04/15 · openalex updated_date 2026/07/28
This paper presents a policy parameterization for learning-based control on nonlinear, partially-observed dynamical systems. The parameterization is based on a nonlinear version of the Youla parameterization and the recently proposed Recurrent Equilibrium Network (REN) class of models. We prove that the resulting Youla-REN parameterization automatically satisfies stability (contraction) and user-tunable robustness (Lipschitz) conditions on the closed-loop system. This means it can be used for safe learning-based control with no additional constraints or projections required to enforce stability or robustness. We test the new policy class in simulation on two reinforcement learning tasks: 1) magnetic suspension, and 2) inverting a rotary-arm pendulum. We find that the Youla-REN performs similarly to existing learning-based and optimal control methods while also ensuring stability and exhibiting improved robustness to adversarial disturbances.