SAFARI: Safe and Active Robot Imitation Learning with Imagination

2020/11/18 by Norman Di Palo, Di Palo, Norman, Edward Johns +1
Computer Science · Engineering · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Algorithms #Reinforcement Learning in Robotics #Robot Manipulation and Learning #Robotics (cs.RO)

paper · pdf · doi:10.48550/arxiv.2011.09586

openalex publication_date 2020/11/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

One of the main issues in Imitation Learning is the erroneous behavior of an agent when facing out-of-distribution situations, not covered by the set of demonstrations given by the expert. In this work, we tackle this problem by introducing a novel active learning and control algorithm, SAFARI. During training, it allows an agent to request further human demonstrations when these out-of-distribution situations are met. At deployment, it combines model-free acting using behavioural cloning with model-based planning to reduce state-distribution shift, using future state reconstruction as a test for state familiarity. We empirically demonstrate how this method increases the performance on a set of manipulation tasks with respect to passive Imitation Learning, by gathering more informative demonstrations and by minimizing state-distribution shift at test time. We also show how this method enables the agent to autonomously predict failure rapidly and safely.

Citations

• A survey of robot learning from demonstration
• End to End Learning for Self-Driving Cars
• Trial without Error: Towards Safe Reinforcement Learning via Human Intervention
• On the exact relationship between the denoising function and the data distribution
• One-Shot Visual Imitation Learning via Meta-Learning
• DropoutDAgger: A Bayesian Approach to Safe Imitation Learning
• Exploration by Random Network Distillation
• Improving Model-Based Control and Active Exploration with Reconstruction Uncertainty Optimization
• Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Traffic
• Learning Latent Plans from Play
• A Reduction of Imitation Learning and Structured Prediction to No-Regret\n Online Learning
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Active Imitation Learning from Multiple Non-Deterministic Teachers: Formulation, Challenges, and Algorithms

Related

• Get Back Here: Robust Imitation by Return-to-Distribution Planning
• HYDRA: Hybrid Robot Actions for Imitation Learning
• Imitator Learning: Achieve Out-of-the-Box Imitation Ability in Variable Environments
• Asking for Help: Failure Prediction in Behavioral Cloning through Value Approximation
• Prepare Before You Act: Learning From Humans to Rearrange Initial States