Hierarchical reinforcement learning for efficient exploration and transfer

2020/11/12 by Lorenzo Steccanella, Simone Totaro, Steccanella, Lorenzo +5 · 1 citation
Biochemistry, Genetics and Molecular Biology · Computer Science · Neuroscience · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural dynamics and brain function #Receptor Mechanisms and Signaling #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2011.06335

openalex publication_date 2020/11/12 · openalex created_date 2020/11/23 · openalex updated_date 2026/07/28

Abstract

Sparse-reward domains are challenging for reinforcement learning algorithms since significant exploration is needed before encountering reward for the first time. Hierarchical reinforcement learning can facilitate exploration by reducing the number of decisions necessary before obtaining a reward. In this paper, we present a novel hierarchical reinforcement learning framework based on the compression of an invariant state space that is common to a range of tasks. The algorithm introduces subtasks which consist of moving between the state partitions induced by the compression. Results indicate that the algorithm can successfully solve complex sparse-reward domains, and transfer knowledge to solve new, previously unseen tasks more quickly.

Citations

• Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
• FeUdal Networks for Hierarchical Reinforcement Learning
• Automatic Goal Generation for Reinforcement Learning Agents
• Hindsight Experience Replay
• Near-Optimal Representation Learning for Hierarchical Reinforcement\n Learning
• Go-Explore: a New Approach for Hard-Exploration Problems

Cited by

• Toward Autonomous UI Exploration: The UIExplorer Benchmark

Related

• Hierarchical Reinforcement Learning By Discovering Intrinsic Options
• Learning and Exploiting Multiple Subgoals for Fast Exploration in Hierarchical Reinforcement Learning
• Learning Representations in Model-Free Hierarchical Reinforcement Learning
• Modulated Policy Hierarchies
• Active Hierarchical Exploration with Stable Subgoal Representation Learning