Q-learning as a monotone scheme

2024/05/30 by Yang, Lingyi
#FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2405.20538

Abstract

Stability issues with reinforcement learning methods persist. To better understand some of these stability and convergence issues involving deep reinforcement learning methods, we examine a simple linear quadratic example. We interpret the convergence criterion of exact Q-learning in the sense of a monotone scheme and discuss consequences of function approximation on monotonicity properties.

Related

• Stabilizing Q-learning with Linear Architectures for Provably Efficient Learning
• An Elementary Proof that Q-learning Converges Almost Surely
• Regularized Q-learning
• Towards Characterizing Divergence in Deep Q-Learning
• Deep Q-Learning: Theoretical Insights from an Asymptotic Analysis