Improving Generalization in Meta-RL with Imaginary Tasks from Latent\n Dynamics Mixture

2021/05/27 by Lee, Suyoung, Sae-Young Chung, Chung, Sae-Young · 2 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification #Multimodal Machine Learning Applications #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2105.13524

openalex publication_date 2021/05/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The generalization ability of most meta-reinforcement learning (meta-RL)\nmethods is largely limited to test tasks that are sampled from the same\ndistribution used to sample training tasks. To overcome the limitation, we\npropose Latent Dynamics Mixture (LDM) that trains a reinforcement learning\nagent with imaginary tasks generated from mixtures of learned latent dynamics.\nBy training a policy on mixture tasks along with original training tasks, LDM\nallows the agent to prepare for unseen test tasks during training and prevents\nthe agent from overfitting the training tasks. LDM significantly outperforms\nstandard meta-RL methods in test returns on the gridworld navigation and MuJoCo\ntasks where we strictly separate the training task distribution and the test\ntask distribution.\n

Citations

• Auto-Encoding Variational Bayes
• Planning and acting in partially observable stochastic domains
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• A Study on Overfitting in Deep Reinforcement Learning
• Unsupervised Meta-Learning for Reinforcement Learning
• Virtual Class Enhanced Discriminative Embedding Learning
• Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions
• Fast Context Adaptation via Meta-Learning
• Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic\n Context Variables
• Meta reinforcement learning as task inference
• Generalization in Reinforcement Learning with Selective Noise Injection\n and Information Bottleneck
• Unsupervised Curricula for Visual Meta-Reinforcement Learning
• Improving Generalization in Meta Reinforcement Learning using Learned Objectives
• Curriculum in Gradient-Based Meta-Reinforcement Learning
• Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutions
• Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learning
• Provably Efficient Model-based Policy Adaptation
• Meta-Reinforcement Learning Robust to Distributional Shift via Model Identification and Experience Relabeling
• Deep Reinforcement Learning amidst Lifelong Non-Stationarity
• Automatic Data Augmentation for Generalization in Deep Reinforcement Learning
• Linear Representation Meta-Reinforcement Learning for Instant Adaptation

Cited by

• An Information-Theoretic Analysis of Out-of-Distribution Generalization in Meta-Learning with Applications to Meta-RL
• Zero-Shot Context Generalization in Reinforcement Learning from Few Training Contexts

Related

• Meta-Reinforcement Learning in Broad and Non-Parametric Environments
• TIMRL: A Novel Meta-Reinforcement Learning Framework for Non-Stationary and Multi-Task Environments
• Meta-Reinforcement Learning Robust to Distributional Shift via Model Identification and Experience Relabeling
• Task-Aware Virtual Training: Enhancing Generalization in Meta-Reinforcement Learning for Out-of-Distribution Tasks
• Dream to Adapt: Meta Reinforcement Learning by Latent Context Imagination and MDP Imagination