2020/02/18 by Sharath Chandra Raparthy, Bhairav Mehta, Raparthy, Sharath Chandra +5 · 5 citations
Computer Science · Engineering · Mathematics · Psychology · #Artificial Intelligence (cs.AI) #Artificial intelligence #Computer science #Curriculum #Domain (mathematical analysis) #Domain Adaptation and Few-Shot Learning #Engineering #FOS: Computer and information sciences #Human–computer interaction #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine learning #Programming language #Psychology #Reinforcement Learning in Robotics #Reinforcement learning #Robot Manipulation and Learning #Robotics (cs.RO) #Set (abstract data type) #Task (project management) #cs.AI #cs.LG #cs.RO #stat.ML
paper · pdf · doi:10.48550/arxiv.2002.07911
published in arXiv (Cornell University) (Cornell University)
arxiv created 2020/10/26 · arxiv updated 2020/10/28
Goal-directed Reinforcement Learning (RL) traditionally considers an agent interacting with an environment, prescribing a real-valued reward to an agent proportional to the completion of some goal. Goal-directed RL has seen large gains in sample efficiency, due to the ease of reusing or generating new experience by proposing goals. One approach,self-play, allows an agent to "play" against itself by alternatively setting and accomplishing goals, creating a learned curriculum through which an agent can learn to accomplish progressively more difficult goals. However, self-play has been limited to goal curriculum learning or learning progressively harder goals within a single environment. Recent work on robotic agents has shown that varying the environment during training, for example with domain randomization, leads to more robust transfer. As a result, we extend the self-play framework to jointly learn a goal and environment curriculum, leading to an approach that learns the most fruitful domain randomization strategy with self-play. Our method, Self-Supervised Active Domain Randomization(SS-ADR), generates a coupled goal-task curriculum, where agents learn through progressively more difficult tasks and environment variations. By encouraging the agent to try tasks that are just outside of its current capabilities, SS-ADR builds a domain randomization curriculum that enables state-of-the-art results on varioussim2real transfer tasks. Our results show that a curriculum of co-evolving the environment difficulty together with the difficulty of goals set in each environment provides practical benefits in the goal-directed tasks tested.