Reinforcement Learning for a Discrete-Time Linear-Quadratic Control Problem with an Application

2024/12/08 by Liang Li, Li, Lucky
Computer Science · #Adaptive Dynamic Programming Control #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC)

paper · pdf · doi:10.48550/arxiv.2412.05906

openalex publication_date 2024/12/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We study the discrete-time linear-quadratic (LQ) control model using reinforcement learning (RL). Using entropy to measure the cost of exploration, we prove that the optimal feedback policy for the problem must be Gaussian type. Then, we apply the results of the discrete-time LQ model to solve the discrete-time mean-variance asset-liability management problem and prove our RL algorithm's policy improvement and convergence. Finally, a numerical example sheds light on the theoretical results established using simulations.

Related

• Stochastic Linear Quadratic Optimal Control Problem: A Reinforcement Learning Method
• A Tour of Reinforcement Learning: The View from Continuous Control
• Robust Reinforcement Learning for Risk-Sensitive Linear Quadratic Gaussian Control
• Discrete-Time Mean-Variance Strategy Based on Reinforcement Learning
• Robust Reinforcement Learning: A Case Study in Linear Quadratic Regulation