Bayesian Model-Agnostic Meta-Learning

2018/06/11 by Taesup Kim, Kim, Taesup, Jaesik Yoon +9 · 15 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.1806.03836

openalex publication_date 2018/06/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Learning to infer Bayesian posterior from a few-shot dataset is an important step towards robust meta-learning due to the model uncertainty inherent in the problem. In this paper, we propose a novel Bayesian model-agnostic meta-learning method. The proposed method combines scalable gradient-based meta-learning with nonparametric variational inference in a principled probabilistic framework. During fast adaptation, the method is capable of learning complex uncertainty structure beyond a point estimate or a simple Gaussian approximation. In addition, a robust Bayesian meta-update mechanism with a new meta-loss prevents overfitting during meta-update. Remaining an efficient gradient-based meta-learner, the method is also model-agnostic and simple to implement. Experiment results show the accuracy and robustness of the proposed method in various tasks: sinusoidal regression, image classification, active learning, and reinforcement learning.

Citations

• Optimizing Neural Networks with Kronecker-factored Approximate Curvature
• Stein Variational Gradient Descent: A General Purpose Bayesian Inference\n Algorithm
• RL²: Fast Reinforcement Learning via Slow Reinforcement Learning
• Stein Variational Policy Gradient
• Recasting Gradient-Based Meta-Learning as Hierarchical Bayes

Cited by

• MAD-NG: Meta-Auto-Decoder Neural Galerkin Method for Solving Parametric Partial Differential Equations
• Generalised Linear Models in Deep Bayesian RL with Learnable Basis Functions
• Context-aware Learned Mesh-based Simulation via Trajectory-Level Meta-Learning
• MetaKoopman: Bayesian Meta-Learning of Koopman Operators for Modeling Structured Dynamics under Distribution Shifts
• Compositional ADAM: An Adaptive Compositional Solver
• A review of uncertainty quantification in deep learning: Techniques, applications and challenges
• Neural Variational Dropout Processes
• MaNGO - Adaptable Graph Network Simulators via Meta-Learning
• The effects of negative adaptation in Model-Agnostic Meta-Learning
• On Quantification of Borrowing of Information in Hierarchical Bayesian Models
• Adaptive Deep Kernel Learning
• Learnable Loss Geometries with Mirror Descent for Scalable and Convergent Meta-Learning
• Optimizing In-Context Learning for Efficient Full Conformal Prediction
• Self-organization of action hierarchy and compositionality by reinforcement learning with recurrent neural networks
• Domain-Generalization to Improve Learning in Meta-Learning Algorithms

Related

• Gradient-EM Bayesian Meta-learning
• Probabilistic Model-Agnostic Meta-Learning
• Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
• Bayesian Meta-Learning Through Variational Gaussian Processes
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks