Learning by Playing - Solving Sparse Reward Tasks from Scratch

2018/02/28 by Riedmiller, Martin, Hafner, Roland, Lampe, Thomas +6 · 5 citations
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Robotics (cs.RO)

paper · doi:10.48550/arxiv.1802.10567

Abstract

We propose Scheduled Auxiliary Control (SAC-X), a new learning paradigm in the context of Reinforcement Learning (RL). SAC-X enables learning of complex behaviors - from scratch - in the presence of multiple sparse reward signals. To this end, the agent is equipped with a set of general auxiliary tasks, that it attempts to learn simultaneously via off-policy RL. The key idea behind our method is that active (learned) scheduling and execution of auxiliary policies allows the agent to efficiently explore its environment - enabling it to excel at sparse reward RL. Our experiments in several challenging robotic manipulation settings demonstrate the power of our approach.

Cited by

• Learning When to Switch: Adaptive Policy Selection via Reinforcement Learning
• Reinforcement learning of quantum circuit architectures for molecular potential energy curves
• Learning Parameterized Skills from Demonstrations
• Generating Auxiliary Tasks with Reinforcement Learning
• SparseMap: A Sparse Tensor Accelerator Framework Based on Evolution Strategy

Related

• Dealing with Sparse Rewards in Reinforcement Learning
• Reinforcement Learning via Auxiliary Task Distillation
• Reinforcement Learning with Unsupervised Auxiliary Tasks
• Dense Dynamics-Aware Reward Synthesis: Integrating Prior Experience with Demonstrations
• Revisiting Sparse Rewards for Goal-Reaching Reinforcement Learning