Natural Evolution Strategies as a Black Box Estimator for Stochastic Variational Inference

2023/08/15 by Ahmad Ayaz Amin, Amin, Ahmad Ayaz
Computer Science · #Gaussian Processes and Bayesian Inference #Generative Adversarial Networks and Image Synthesis #Machine Learning and Algorithms

paper · pdf · doi:10.48550/arxiv.2308.08053

Abstract

Stochastic variational inference and its derivatives in the form of variational autoencoders enjoy the ability to perform Bayesian inference on large datasets in an efficient manner. However, performing inference with a VAE requires a certain design choice (i.e. reparameterization trick) to allow unbiased and low variance gradient estimation, restricting the types of models that can be created. To overcome this challenge, an alternative estimator based on natural evolution strategies is proposed. This estimator does not make assumptions about the kind of distributions used, allowing for the creation of models that would otherwise not have been possible under the VAE framework.

Related

• Two Methods For Wild Variational Inference
• Training Variational Autoencoders with Buffered Stochastic Variational Inference
• Quasi Black-Box Variational Inference with Natural Gradients for Bayesian Learning
• Natural Gradient Variational Inference with Gaussian Mixture Models
• Smoothed Gradients for Stochastic Variational Inference