Imitating Latent Policies from Observation

2018/05/21 by Ashley D. Edwards, Himanshu Sahni, Edwards, Ashley D. +5 · 13 citations
Computer Science · Mathematics · #FOS: Computer and information sciences #Human Pose and Action Recognition #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Multimodal Machine Learning Applications #Reinforcement Learning in Robotics #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1805.07914

Accepted to ICML 2019

openalex publication_date 2018/05/21 · arxiv created 2019/05/13 · arxiv updated 2019/05/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we describe a novel approach to imitation learning that infers latent policies directly from state observations. We introduce a method that characterizes the causal effects of latent actions on observations while simultaneously predicting their likelihood. We then outline an action alignment procedure that leverages a small amount of environment interactions to determine a mapping between the latent and real-world actions. We show that this corrected labeling can be used for imitating the observed behavior, even though no expert actions are given. We evaluate our approach within classic control environments and a platform game and demonstrate that it performs better than standard approaches. Code for this work is available at https://github.com/ashedwards/ILPO.

Citations

• Action-Conditional Video Prediction using Deep Networks in Atari Games
• Third-Person Imitation Learning
• Recurrent Environment Simulators
• AI2-THOR: An Interactive 3D Environment for Visual AI
• Forward-Backward Reinforcement Learning
• Recall Traces: Backtracking Models for Efficient Reinforcement Learning
• Quantifying Generalization in Reinforcement Learning
• Time-Contrastive Networks: Self-Supervised Learning from Video
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets

Cited by

• Embodied Robot Manipulation in the Era of Foundation Models: Planning and Learning Perspectives
• Videos are Sample-Efficient Supervisions: Behavior Cloning from Videos via Latent Representations
• Motus: A Unified Latent Action World Model
• Closing the Train-Test Gap in World Models for Gradient-Based Planning
• LAOF: Robust Latent Action Learning with Optical Flow Constraints
• Improving Robustness to Out-of-Distribution States in Imitation Learning via Deep Koopman-Boosted Diffusion Policy
• TimeRewarder: Learning Dense Reward from Passive Videos via Frame-wise Temporal Distance
• ShadowDancer: Teaching Video World Models Any Action by Learning Unified Dynamics Representations from a Video and Its Shadow
• Learning from Observation: A Survey of Recent Advances
• VITA: Zero-Shot Value Functions via Test-Time Adaptation of Vision-Language Models
• What Do Latent Action Models Actually Learn?
• Learning Latent Action World Models In The Wild
• Latent Diffusion Planning for Imitation Learning

Related

• Causal Imitation Learning under Temporally Correlated Noise
• Learning from Observation: A Survey of Recent Advances
• Mimicking Better by Matching the Approximate Action Distribution
• Augmented Behavioral Cloning from Observation
• Imitation Learning from Video by Leveraging Proprioception