2023/12/19 by Tomonori Sadamoto, Sadamoto, Tomonori, Fumiya Nakamata +1
Computer Science · Decision Sciences · Engineering · #Adaptive Dynamic Programming Control #Advanced Bandit Algorithms Research #Advanced Control Systems Optimization #FOS: Electrical engineering #FOS: Mathematics #Optimization and Control (math.OC) #Systems and Control (eess.SY) #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2312.12173
openalex publication_date 2023/12/19 · openalex created_date 2023/12/22 · openalex updated_date 2026/07/28
We present a model-based globally convergent policy gradient method (PGM) for linear quadratic Gaussian (LQG) control. Firstly, we establish equivalence between optimizing dynamic output feedback controllers and designing a static feedback gain for a system represented by a finite-length input-output history (IOH). This IOH-based approach allows us to explore LQG controllers within a parameter space defined by IOH gains. Secondly, by considering a control law comprising the IOH gain and a sufficiently small random perturbation, we show that the cost function, evaluated through the control law over IOH gains, is gradient-dominant and locally smooth, ensuring the global linear convergence of the PGM. Numerical simulations show that the dynamic controller learned by the proposed PGM is almost same as the LQG optimal controller, indicating promising results even in a reduced-order controller design.