Intrinsic Rewards from Self-Organizing Feature Maps for Exploration in Reinforcement Learning

2023/02/06 by Marius Lindegaard, Lindegaard, Marius, Hjalmar Jacob Vinje +3
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.2302.04125

openalex publication_date 2023/02/06 · openalex created_date 2023/02/11 · openalex updated_date 2026/07/28

Abstract

We introduce an exploration bonus for deep reinforcement learning methods calculated using self-organising feature maps. Our method uses adaptive resonance theory (ART) providing online, unsupervised clustering to quantify the novelty of a state. This heuristic is used to add an intrinsic reward to the extrinsic reward signal for then to optimize the agent to maximize the sum of these two rewards. We find that this method was able to play the game Ordeal at a human level after a comparable number of training epochs to ICM arXiv:1705.05464. Agents augmented with RND arXiv:1810.12894 were unable to achieve the same level of performance in our space of hyperparameters.

Citations

• Diseña, Fabrica y Programa Tu Propio Robot
• Exploration by Random Network Distillation

Related

• Intrinsically-Motivated Reinforcement Learning: A Brief Introduction
• Successor-Predecessor Intrinsic Exploration
• Perturbation-based exploration methods in deep reinforcement learning
• Autonomous state-space segmentation for Deep-RL sparse reward scenarios
• Never Give Up: Learning Directed Exploration Strategies