Amortised Inference in Neural Networks for Small-Scale Probabilistic Meta-Learning

2023/10/24 by Matthew Ashman, Ashman, Matthew, Tommy Rochussen +3 · 2 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2310.15786

openalex publication_date 2023/10/24 · openalex created_date 2023/10/26 · openalex updated_date 2026/07/28

Abstract

The global inducing point variational approximation for BNNs is based on using a set of inducing inputs to construct a series of conditional distributions that accurately approximate the conditionals of the true posterior distribution. Our key insight is that these inducing inputs can be replaced by the actual data, such that the variational distribution consists of a set of approximate likelihoods for each datapoint. This structure lends itself to amortised inference, in which the parameters of each approximate likelihood are obtained by passing each datapoint through a meta-model known as the inference network. By training this inference network across related datasets, we can meta-learn Bayesian inference over task-specific BNNs.

Cited by

• Amortising Inference and Meta-Learning Priors in Neural Networks
• Weight-Space Linear Recurrent Neural Networks

Related

• Amortised Inference in Bayesian Neural Networks
• Amortising Inference and Meta-Learning Priors in Neural Networks
• Meta-Learning with Shared Amortized Variational Inference
• Meta-Amortized Variational Inference and Learning
• Neural Methods for Amortized Inference