Robust Recurrence of Discrete-Time Infinite-Horizon Stochastic Optimal Control with Discounted Cost

2025/04/29 by Moldenhauer, Robert H., Nešić, Dragan, Granzotto, Mathieu +2
#93D09 #93D30 #93E20 #FOS: Electrical engineering #FOS: Mathematics #Optimization and Control (math.OC) #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2504.20705

Abstract

We analyze the stability of general nonlinear discrete-time stochastic systems controlled by optimal inputs that minimize an infinite-horizon discounted cost. Under a novel stochastic formulation of cost-controllability and detectability assumptions inspired by the related literature on deterministic systems, we prove that uniform semi-global practical recurrence holds for the closed-loop system, where the adjustable parameter is the discount factor. Under additional continuity assumptions, we further prove that this property is robust.

Related

• Stability analysis of optimal control problems with time-dependent costs
• Robustness to Incorrect Models and Data-Driven Learning in Average-Cost Optimal Stochastic Control
• Robustness of Stochastic Optimal Control to Approximate Diffusion Models under Several Cost Evaluation Criteria
• Policy iteration for discrete-time systems with discounted costs: stability and near-optimality guarantees
• Asymptotic Analysis of Empirical Dynamic Programming in Infinite-Horizon Stochastic Optimal Control