Multi-task Reinforcement Learning with a Planning Quasi-Metric

2020/02/08 by Vincent Micheli, Micheli, Vincent, Karthigan Sinnathamby +3
Computer Science · Engineering · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics #Robotic Path Planning Algorithms #Scheduling and Optimization Algorithms

paper · pdf · doi:10.48550/arxiv.2002.03240

openalex publication_date 2020/02/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We introduce a new reinforcement learning approach combining a planning quasi-metric (PQM) that estimates the number of steps required to go from any state to another, with task-specific "aimers" that compute a target state to reach a given goal. This decomposition allows the sharing across tasks of a task-agnostic model of the quasi-metric that captures the environment's dynamics and can be learned in a dense and unsupervised manner. We achieve multiple-fold training speed-up compared to recently published methods on the standard bit-flip problem and in the MuJoCo robotic arm simulator.

Citations

• Continuous control with deep reinforcement learning
• Hindsight Experience Replay
• Multi-Goal Reinforcement Learning: Challenging Robotics Environments and\n Request for Research
• Multi-task Deep Reinforcement Learning with PopArt
• Planning with Goal-Conditioned Policies
• PyTorch: An Imperative Style, High-Performance Deep Learning Library

Related

• Enhancing Robotic Manipulation: Harnessing the Power of Multi-Task Reinforcement Learning and Single Life Reinforcement Learning in Meta-World
• Floyd-Warshall Reinforcement Learning: Learning from Past Experiences to Reach New Goals
• PlanGAN: Model-based Planning With Sparse Rewards and Multiple Goals
• Offline Goal-Conditioned Reinforcement Learning with Projective Quasimetric Planning
• Multistep Quasimetric Learning for Scalable Goal-conditioned Reinforcement Learning