Efficient Exploration through Intrinsic Motivation Learning for Unsupervised Subgoal Discovery in Model-Free Hierarchical Reinforcement Learning

2019/11/18 by Jacob Rafati, Rafati, Jacob, David C. Noelle +1
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Metaheuristic Optimization Algorithms Research #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.1911.10164

openalex publication_date 2019/11/18 · openalex created_date 2019/12/05 · openalex updated_date 2026/07/28

Abstract

Efficient exploration for automatic subgoal discovery is a challenging problem in Hierarchical Reinforcement Learning (HRL). In this paper, we show that intrinsic motivation learning increases the efficiency of exploration, leading to successful subgoal discovery. We introduce a model-free subgoal discovery method based on unsupervised learning over a limited memory of agent's experiences during intrinsic motivation. Additionally, we offer a unified approach to learning representations in model-free HRL.

Citations

• Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
• Surprise-Based Intrinsic Motivation for Deep Reinforcement Learning

Related

• Learning Representations in Model-Free Hierarchical Reinforcement Learning
• Active Hierarchical Exploration with Stable Subgoal Representation Learning
• Learning and Exploiting Multiple Subgoals for Fast Exploration in Hierarchical Reinforcement Learning
• Hierarchical Reinforcement Learning By Discovering Intrinsic Options
• Intrinsically-Motivated Reinforcement Learning: A Brief Introduction