Curiosity-Driven Multi-Criteria Hindsight Experience Replay

2019/06/09 by John B. Lanier, Stephen McAleer, Lanier, John B. +3 · 3 citations
Computer Science · Mathematics · Psychology · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Psychological and Educational Research Studies #Robotics (cs.RO) #Social Robot Interaction and HRI #Visual Attention and Saliency Detection #cs.AI #cs.LG #cs.RO #stat.ML

paper · pdf · doi:10.48550/arxiv.1906.03710

14 pages

arxiv created 2019/06/09 · openalex publication_date 2019/06/09 · arxiv updated 2019/06/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

Dealing with sparse rewards is a longstanding challenge in reinforcement learning. The recent use of hindsight methods have achieved success on a variety of sparse-reward tasks, but they fail on complex tasks such as stacking multiple blocks with a robot arm in simulation. Curiosity-driven exploration using the prediction error of a learned dynamics model as an intrinsic reward has been shown to be effective for exploring a number of sparse-reward environments. We present a method that combines hindsight with curiosity-driven exploration and curriculum learning in order to solve the challenging sparse-reward block stacking task. We are the first to stack more than two blocks using only sparse reward without human demonstrations.

Citations

• Continuous control with deep reinforcement learning
• Variational Intrinsic Control
• Parameter Space Noise for Exploration
• Reverse Curriculum Generation for Reinforcement Learning
• Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research
• Exploration by Random Network Distillation
• Go-Explore: a New Approach for Hard-Exploration Problems

Cited by

• Uncertainty-Based Smooth Policy Regularisation for Reinforcement Learning with Few Demonstrations
• Effects of sparse rewards of different magnitudes in the speed of learning of model-based actor critic methods
• Learning Compositional Neural Programs for Continuous Control

Related

• Hindsight Experience Replay
• Imaginary Hindsight Experience Replay: Curious Model-based Learning for Sparse Reward Tasks
• Improvements on Hindsight Learning
• Competitive Experience Replay
• Learning and reusing primitive behaviours to improve Hindsight Experience Replay sample efficiency