Reinforcement and Imitation Learning for Diverse Visuomotor Skills

2018/02/26 by Yuke Zhu, Zhu, Yuke, Ziyu Wang +19 · 5 citations
Computer Science · Engineering · #Advanced Vision and Imaging #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics #Robot Manipulation and Learning #Robotics (cs.RO)

paper · pdf · doi:10.48550/arxiv.1802.09564

openalex publication_date 2018/02/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose a model-free deep reinforcement learning method that leverages a small amount of demonstration data to assist a reinforcement learning agent. We apply this approach to robotic manipulation tasks and train end-to-end visuomotor policies that map directly from RGB camera inputs to joint velocities. We demonstrate that our approach can solve a wide variety of visuomotor tasks, for which engineering a scripted controller would be laborious. In experiments, our reinforcement and imitation agent achieves significantly better performances than agents trained with reinforcement learning or imitation learning alone. We also illustrate that these policies, trained with large visual and dynamics variations, can achieve preliminary successes in zero-shot sim2real transfer. A brief visual description of this work can be viewed in https://youtu.be/EDl8SQUNjj0

Citations

• Adam: A Method for Stochastic Optimization
• Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization
• Learning Dexterous Manipulation for a Soft Robotic Hand from Human Demonstration
• Generative Adversarial Imitation Learning
• One-Shot Imitation Learning
• Transferring End-to-End Visuomotor Control from Simulation to Real World\n for a Multi-Stage Task
• Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation
• One-Shot Visual Imitation Learning via Meta-Learning
• Time-Contrastive Networks: Self-Supervised Learning from Video
• Domain Randomization for Transferring Deep Neural Networks from\n Simulation to the Real World
• A Reduction of Imitation Learning and Structured Prediction to No-Regret\n Online Learning

Cited by

• Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfer
• VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation
• Confounding Robust Deep Reinforcement Learning: A Causal Approach
• ImaginationPolicy: Towards Generalizable, Precise and Reliable End-to-End Policy for Robotic Manipulation
• ROVER: Recursive Reasoning Over Videos with Vision-Language Models for Embodied Tasks

Related

• Adversarial Feature Training for Generalizable Robotic Visuomotor Control
• Goal-Conditioned End-to-End Visuomotor Control for Versatile Skill Primitives
• Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstration
• Imitation Learning-based Direct Visual Servoing using the Large Projection Formulation
• Visual Imitation Learning of Non-Prehensile Manipulation Tasks with Dynamics-Supervised Models