Reinforcement Learning for Stochastic LQ Control of Discrete-Time Systems with Multiplicative Noises

2023/11/21 by Hongdan Li, Lucky Qiaofeng Li, Li, Hongdan +5
Computer Science · Engineering · #Adaptive Dynamic Programming Control #Extremum Seeking Control Systems #FOS: Mathematics #Optimization and Control (math.OC) #Traffic control and management

paper · pdf · doi:10.48550/arxiv.2311.12322

openalex publication_date 2023/11/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper considers a stochastic linear quadratic problem for discrete-time systems with multiplicative noises over an infinite horizon. To obtain the optimal solution, we propose an online iterative algorithm of reinforcement learning based on Bellman dynamic programming principle. The algorithm avoids the direct calculation of algebra Riccati equations. It merely takes advantage of state trajectories over a short interval instead of all iterations, significantly simplifying the calculation process. Under the stabilizable initial values, numerical examples shed light on our theoretical results.

Related

• Stochastic Linear Quadratic Optimal Control Problem: A Reinforcement Learning Method
• Average Cost Optimal Control of Stochastic Systems Using Reinforcement Learning
• Model-free optimal control of discrete-time systems with additive and multiplicative noises
• Model-free stochastic linear quadratic control for discrete-time systems with multiplicative and additive noises via semidefinite programming
• Reinforcement Learning for Adaptive Optimal Stationary Control of Linear Stochastic Systems