Reannealing of Decaying Exploration Based On Heuristic Measure in Deep Q-Network

2020/09/29 by Xing Wang, Wang, Xing, Alexander Vinel +1
Computer Science · #Artificial Intelligence (cs.AI) #Distributed and Parallel Computing Systems #FOS: Computer and information sciences #Reinforcement Learning in Robotics #Robotic Path Planning Algorithms

paper · pdf · doi:10.48550/arxiv.2009.14297

openalex publication_date 2020/09/29 · openalex created_date 2020/10/08 · openalex updated_date 2026/07/28

Abstract

Existing exploration strategies in reinforcement learning (RL) often either ignore the history or feedback of search, or are complicated to implement. There is also a very limited literature showing their effectiveness over diverse domains. We propose an algorithm based on the idea of reannealing, that aims at encouraging exploration only when it is needed, for example, when the algorithm detects that the agent is stuck in a local optimum. The approach is simple to implement. We perform an illustrative case study showing that it has potential to both accelerate training and obtain a better policy.

Citations

• Adam: A Method for Stochastic Optimization
• #Exploration: A Study of Count-Based Exploration for Deep Reinforcement\n Learning

Related

• DQN with model-based exploration: efficient learning on environments with sparse rewards
• Explicit Explore-Exploit Algorithms in Continuous State Spaces
• Depth and nonlinearity induce implicit exploration for RL
• Diversity-Driven Exploration Strategy for Deep Reinforcement Learning
• The many faces of optimism - Extended version