Hyperbolic Embeddings for Learning Options in Hierarchical Reinforcement Learning

2018/12/04 by Saket Tiwari, Tiwari, Saket, M. Prannoy +1
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Human Pose and Action Recognition #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.1812.01487

openalex publication_date 2018/12/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Hierarchical reinforcement learning deals with the problem of breaking down large tasks into meaningful sub-tasks. Autonomous discovery of these sub-tasks has remained a challenging problem. We propose a novel method of learning sub-tasks by combining paradigms of routing in computer networks and graph based skill discovery within the options framework to define meaningful sub-goals. We apply the recent advancements of learning embeddings using Riemannian optimisation in the hyperbolic space to embed the state set into the hyperbolic space and create a model of the environment. In doing so we enforce a global topology on the states and are able to exploit this topology to learn meaningful sub-tasks. We demonstrate empirically, both in discrete and continuous domains, how these embeddings can improve the learning of meaningful sub-tasks.

Citations

• Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation
• Meta Learning Shared Hierarchies

Related

• Hyperbolic Deep Reinforcement Learning
• Option Discovery in Hierarchical Reinforcement Learning using Spatio-Temporal Clustering
• Reinforcement Learning in hyperbolic space for multi-step reasoning
• Reinforcement Learning in Hyperbolic Spaces: Models and Experiments
• Hierarchical Reinforcement Learning By Discovering Intrinsic Options