Smooth Q-learning: Accelerate Convergence of Q-learning Using Similarity

2021/06/02 by Liao, Wei, Wei, Xiaohui, Lai, Jizhou
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2106.01134

Abstract

An improvement of Q-learning is proposed in this paper. It is different from classic Q-learning in that the similarity between different states and actions is considered in the proposed method. During the training, a new updating mechanism is used, in which the Q value of the similar state-action pairs are updated synchronously. The proposed method can be used in combination with both tabular Q-learning function and deep Q-learning. And the results of numerical examples illustrate that compared to the classic Q-learning, the proposed method has a significantly better performance.

Related

• Augmented Q Imitation Learning (AQIL)
• Momentum-based Accelerated Q-learning
• Smoothed Q-learning
• Accelerated Target Updates for Q-learning
• An Elementary Proof that Q-learning Converges Almost Surely