Implicit Maximum Likelihood Estimation

2018/09/24 by Ke Li, Jitendra Malik, Li, Ke +1 · 62 citations
Computer Science · Mathematics · Physics and Astronomy · #Applied mathematics #Bayesian Modeling and Causal Inference #Computer science #FOS: Computer and information sciences #Function (biology) #Gaussian Processes and Bayesian Inference #Likelihood function #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Marginal likelihood #Mathematics #Maximum likelihood #Model Reduction and Neural Networks #Neural and Evolutionary Computing (cs.NE) #Parametric model #Parametric statistics #Probabilistic logic #Simple (philosophy) #Statistics #cs.LG #cs.NE #stat.ML

paper · pdf · doi:10.48550/arxiv.1809.09087

published in arXiv (Cornell University) (Cornell University) · 21 pages, 4 figures. In the interest of promoting discussion, we make the reviews available at https://people.eecs.berkeley.edu/~ke.li/papers/imle_reviews.pdf

openalex publication_date 2018/09/24 · arxiv created 2018/10/22 · arxiv updated 2018/10/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Implicit probabilistic models are models defined naturally in terms of a sampling procedure and often induces a likelihood function that cannot be expressed explicitly. We develop a simple method for estimating parameters in implicit models that does not require knowledge of the form of the likelihood function or any derived quantities, but can be shown to be equivalent to maximizing likelihood under some conditions. Our result holds in the non-asymptotic parametric setting, where both the capacity of the model and the number of data examples are finite. We also demonstrate encouraging experimental results.

Citations

• Connectionist learning of belief networks
• Training Products of Experts by Minimizing Contrastive Divergence
• How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?
• Adversarial Feature Learning
• f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
• Improved Techniques for Training GANs
• Energy-based Generative Adversarial Network
• Learning in Implicit Generative Models
• Boundary-Seeking Generative Adversarial Networks
• Training generative neural networks via Maximum Mean Discrepancy optimization
• A Kernel Method for the Two-Sample Problem

Cited by

• LSRS: Latent Scale Rejection Sampling for Visual Autoregressive Modeling
• A Non-Adversarial Approach to Idempotent Generative Modelling
• One Pass Is Not Enough: Recursive Latent Refinement for Generative Models
• Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation
• Increasing the Utility of Synthetic Images through Chamfer Guidance
• A Forest from the Trees: Generation through Neighborhoods
• You Only Need Adversarial Supervision for Semantic Image Synthesis
• Testing for Typicality with Respect to an Ensemble of Learned Distributions
• FakePolisher: Making DeepFakes More Detection-Evasive by Shallow Reconstruction
• Responsible Disclosure of Generative Models Using Scalable Fingerprinting
• Towards A Multi-agent System for Online Hate Speech Detection
• Dodging DeepFake Detection via Implicit Spatial-Domain Notch Filtering
• Inclusive GAN: Improving Data and Minority Coverage in Generative Models
• Super-Resolution via Conditional Implicit Maximum Likelihood Estimation
• Why the Third Axis Is Freedom

Related

• A Note on Using Discretized Simulated Data to Estimate Implicit Likelihoods in Bayesian Analyses
• An Approximate MSE Expression for Maximum Likelihood and Other Implicitly Defined Estimators of Non-Random Parameters (extended version)
• Neural Approximate Sufficient Statistics for Implicit Models
• Besting Good--Turing: Optimality of Non-Parametric Maximum Likelihood for Distribution Estimation
• Implicit MLE: Backpropagating Through Discrete Exponential Family Distributions