Hierarchical Implicit Models and Likelihood-Free Variational Inference

2017/02/28 by Dustin Tran, Rajesh Ranganath, Tran, Dustin +3 · 3 citations
Computer Science · #Bayesian Methods and Mixture Models #Computation (stat.CO) #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Methodology (stat.ME)

paper · pdf · doi:10.48550/arxiv.1702.08896

openalex publication_date 2017/02/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Implicit probabilistic models are a flexible class of models defined by a simulation process for data. They form the basis for theories which encompass our understanding of the physical world. Despite this fundamental nature, the use of implicit models remains limited due to challenges in specifying complex latent structure in them, and in performing inferences in such models with large data sets. In this paper, we first introduce hierarchical implicit models (HIMs). HIMs combine the idea of implicit densities with hierarchical Bayesian modeling, thereby defining models via simulators of data with rich hidden structure. Next, we develop likelihood-free variational inference (LFVI), a scalable variational inference algorithm for HIMs. Key to LFVI is specifying a variational family that is also implicit. This matches the model's flexibility and allows for accurate approximation of the posterior. We demonstrate diverse applications: a large-scale physical simulator for predator-prey populations in ecology; a Bayesian generative adversarial network for discrete data; and a deep implicit model for text generation.

Citations

• On distinguishability criteria for estimating generative models
• Approximating Likelihood Ratios with Calibrated Discriminative Classifiers
• Auto-Encoding Variational Bayes
• Dynamical Bias in the Coin Toss
• Approximate Bayesian computational methods
• Rényi Divergence Variational Inference
• Adversarial Feature Learning
• Learning in Implicit Generative Models
• Edward: A library for probabilistic modeling, inference, and criticism
• GANS for Sequences of Discrete Elements with the Gumbel-softmax\n Distribution
• Training generative neural networks via Maximum Mean Discrepancy optimization
• Weight Uncertainty in Neural Networks
• Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Fast ε-free Inference of Simulation Models with Bayesian Conditional Density Estimation

Cited by

• Mining gold from implicit models to improve likelihood-free inference
• Variational Approaches for Auto-Encoding Generative Adversarial Networks
• Approximate Inference with Amortised MCMC
• Slimmable Generative Adversarial Networks

Related

• Implicit Variational Inference for High-Dimensional Posteriors
• Variational Implicit Processes
• Kernel Implicit Variational Inference
• Bayesian Neural Network Inference via Implicit Models and the Posterior Predictive Distribution
• Hierarchical Variational Models