Learning a Multi-Modal Policy via Imitating Demonstrations with Mixed Behaviors

2019/03/25 by Hsiao, Fang-I, Kuo, Jui-Hsuan, Sun, Min · 1 citation
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.1903.10304

Abstract

We propose a novel approach to train a multi-modal policy from mixed demonstrations without their behavior labels. We develop a method to discover the latent factors of variation in the demonstrations. Specifically, our method is based on the variational autoencoder with a categorical latent variable. The encoder infers discrete latent factors corresponding to different behaviors from demonstrations. The decoder, as a policy, performs the behaviors accordingly. Once learned, the policy is able to reproduce a specific behavior by simply conditioning on a categorical vector. We evaluate our method on three different tasks, including a challenging task with high-dimensional visual inputs. Experimental results show that our approach is better than various baseline methods and competitive with a multi-modal policy trained by ground truth behavior labels.

Cited by

• A Benchmark for Generalizing Across Diverse Team Strategies in Competitive Pokémon

Related

• Burn-In Demonstrations for Multi-Modal Imitation Learning
• Directed-Info GAIL: Learning Hierarchical Policies from Unsegmented Demonstrations using Directed Information
• Masked Imitation Learning: Discovering Environment-Invariant Modalities in Multimodal Demonstrations
• Diverse Imitation Learning via Self-Organizing Generative Models
• Guide Your Agent with Adaptive Multimodal Rewards