2016/09/22 by Coline Devin, Abhishek Gupta, Devin, Coline +7 · 10 citations
Computer Science · Engineering · #Reinforcement Learning in Robotics #Robot Manipulation and Learning #Domain Adaptation and Few-Shot Learning
paper · pdf · doi:10.48550/arxiv.1609.07088
Reinforcement learning (RL) can automate a wide variety of robotic skills,\nbut learning each new skill requires considerable real-world data collection\nand manual representation engineering to design policy classes or features.\nUsing deep reinforcement learning to train general purpose neural network\npolicies alleviates some of the burden of manual representation engineering by\nusing expressive policy classes, but exacerbates the challenge of data\ncollection, since such methods tend to be less efficient than RL with\nlow-dimensional, hand-designed representations. Transfer learning can mitigate\nthis problem by enabling us to transfer information from one skill to another\nand even from one robot to another. We show that neural network policies can be\ndecomposed into "task-specific" and "robot-specific" modules, where the\ntask-specific modules are shared across robots, and the robot-specific modules\nare shared across all tasks on that robot. This allows for sharing task\ninformation, such as perception, between robots and sharing robot information,\nsuch as dynamics and kinematics, between tasks. We exploit this decomposition\nto train mix-and-match modules that can solve new robot-task combinations that\nwere not seen during training. Using a novel neural network architecture, we\ndemonstrate the effectiveness of our transfer method for enabling zero-shot\ngeneralization with a variety of robots and tasks in simulation for both visual\nand non-visual tasks.\n