Relational Mimic for Visual Adversarial Imitation Learning

2019/12/18 by Lionel Blondé, Yichuan Charlie Tang, Blondé, Lionel +5
Computer Science · Mathematics · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Pose and Action Recognition #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics #Robotics (cs.RO) #cs.AI #cs.CV #cs.LG #cs.RO #stat.ML

paper · pdf · doi:10.48550/arxiv.1912.08444

arxiv created 2019/12/18 · openalex publication_date 2019/12/18 · arxiv updated 2019/12/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this work, we introduce a new method for imitation learning from video demonstrations. Our method, Relational Mimic (RM), improves on previous visual imitation learning methods by combining generative adversarial networks and relational learning. RM is flexible and can be used in conjunction with other recent advances in generative adversarial imitation learning to better address the need for more robust and sample-efficient approaches. In addition, we introduce a new neural network architecture that improves upon the previous state-of-the-art in reinforcement learning and illustrate how increasing the relational reasoning capabilities of the agent enables the latter to achieve increasingly higher performance in a challenging locomotion task with pixel inputs. Finally, we study the effects and contributions of relational learning in policy evaluation, policy improvement and reward learning through ablation studies.

Citations

• A survey of robot learning from demonstration
• Dueling Network Architectures for Deep Reinforcement Learning
• End to End Learning for Self-Driving Cars
• Inverse Reward Design
• Are GANs Created Equal? A Large-Scale Study
• AI Safety Gridworlds
• IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
• Synthesizing Programs for Images using Reinforced Adversarial Learning
• Self-Attention Generative Adversarial Networks
• Risk-Sensitive Generative Adversarial Imitation Learning
• Scalable agent alignment via reward modeling: a research direction
• Quantifying Generalization in Reinforcement Learning
• Deep Residual Learning for Image Recognition
• Stand-Alone Self-Attention in Vision Models
• A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
• Deep Reinforcement Learning that Matters
• Learning dexterous in-hand manipulation

Related

• Visual Adversarial Imitation Learning using Variational Models
• Graph-Structured Visual Imitation
• Injective State-Image Mapping facilitates Visual Adversarial Imitation Learning
• An Introduction to Deep Reinforcement and Imitation Learning
• Visually Robust Adversarial Imitation Learning from Videos with Contrastive Learning