Convergence of Dynamic Programming on the Semidefinite Cone

2021/06/17 by Lee, Donghwan · 2 citations
#FOS: Electrical engineering #FOS: Mathematics #Optimization and Control (math.OC) #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2106.09391

Abstract

The goal of this paper is to investigate new and simple convergence analysis of dynamic programming for linear quadratic regulator problem of discrete-time linear time-invariant systems. In particular, bounds on errors are given in terms of both matrix inequalities and matrix norm. Under a mild assumption on the initial parameter, we prove that the Q-value iteration exponentially converges to the optimal solution. Moreover, a global asymptotic convergence is also presented. These results are then extended to the policy iteration. We prove that in contrast to the Q-value iteration, the policy iteration always converges exponentially fast. An example is given to illustrate the results.

Cited by

• Lyapunov and Riccati Equations from a Positive System Perspective
• A Fully Data-Driven Value Iteration for Stochastic LQR: Convergence, Robustness and Stability

Related

• Convergence and Robustness of Value and Policy Iteration for the Linear Quadratic Regulator
• On Global and Local Convergence of Iterative Linear Quadratic Optimization Algorithms for Discrete Time Nonlinear Control
• Performance guarantees for model-based Approximate Dynamic Programming\n in continuous spaces
• On the Convergence of Policy Iteration in Stationary Dynamic Programming
• Stable Optimal Control and Semicontractive Dynamic Programming