Reinforcement Learning for Adaptive Optimal Stationary Control of Linear Stochastic Systems

2021/07/16 by Bo Pang, Pang, Bo, Zhong‐Ping Jiang +1 · 4 citations
Computer Science · Engineering · #Adaptive Dynamic Programming Control #Smart Grid Energy Management #Energy Load and Power Forecasting

paper · pdf · doi:10.48550/arxiv.2107.07788

Abstract

This paper studies the adaptive optimal stationary control of continuous-time linear stochastic systems with both additive and multiplicative noises, using reinforcement learning techniques. Based on policy iteration, a novel off-policy reinforcement learning algorithm, named optimistic least-squares-based policy iteration, is proposed which is able to find iteratively near-optimal policies of the adaptive optimal stationary control problem directly from input/state data without explicitly identifying any system matrices, starting from an initial admissible control policy. The solutions given by the proposed optimistic least-squares-based policy iteration are proved to converge to a small neighborhood of the optimal solution with probability one, under mild conditions. The application of the proposed algorithm to a triple inverted pendulum example validates its feasibility and effectiveness.

Cited by

• Model-free stochastic linear quadratic control for discrete-time systems with multiplicative and additive noises via semidefinite programming
• Comparative Study of Q-Learning for State-Feedback LQG Control with an Unknown Model
• Learning-based primal-dual optimal control of discrete-time stochastic systems with multiplicative noise
• A Fully Data-Driven Value Iteration for Stochastic LQR: Convergence, Robustness and Stability

Related

• Reinforcement Learning for Stochastic LQ Control of Discrete-Time Systems with Multiplicative Noises
• Model-free optimal control of discrete-time systems with additive and multiplicative noises
• Technical Report: Adaptive Control for Linearizable Systems Using On-Policy Reinforcement Learning
• Stochastic Linear Quadratic Optimal Control Problem: A Reinforcement Learning Method
• Average Cost Optimal Control of Stochastic Systems Using Reinforcement Learning