Vision-Based Multi-Task Manipulation for Inexpensive Robots Using\n End-To-End Learning from Demonstration

2017/07/10 by Rouhollah Rahmatizadeh, Pooya Abolghasemi, Rahmatizadeh, Rouhollah +5 · 5 citations
Engineering · Computer Science · #Robot Manipulation and Learning #Human Pose and Action Recognition #Domain Adaptation and Few-Shot Learning

paper · pdf · doi:10.48550/arxiv.1707.02920

Abstract

We propose a technique for multi-task learning from demonstration that trains\nthe controller of a low-cost robotic arm to accomplish several complex picking\nand placing tasks, as well as non-prehensile manipulation. The controller is a\nrecurrent neural network using raw images as input and generating robot arm\ntrajectories, with the parameters shared across the tasks. The controller also\ncombines VAE-GAN-based reconstruction with autoregressive multimodal action\nprediction. Our results demonstrate that it is possible to learn complex\nmanipulation tasks, such as picking up a towel, wiping an object, and\ndepositing the towel to its previous position, entirely from raw images with\ndirect behavior cloning. We show that weight sharing and reconstruction-based\nregularization substantially improve generalization and robustness, and\ntraining on multiple tasks simultaneously increases the success rate on all\ntasks.\n

Cited by

• Learning Semantic Atomic Skills for Multi-Task Robotic Manipulation
• Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuning
• State-Conditional Adversarial Learning: An Off-Policy Visual Domain Transfer Method for End-to-End Imitation Learning
• Failure Prediction at Runtime for Generative Robot Policies
• Behavioral Exploration: Learning to Explore via In-Context Adaptation

Related

• Learning Sensorimotor Primitives of Sequential Manipulation Tasks from Visual Demonstrations
• A Vision-Guided Multi-Robot Cooperation Framework for Learning-by-Demonstration and Task Reproduction
• Task-Embedded Control Networks for Few-Shot Imitation Learning
• Vision-based Robot Manipulation Learning via Human Demonstrations
• QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation