Truncated Inference for Latent Variable Optimization Problems: Application to Robust Estimation and Learning

2020/03/12 by Christopher Zach, Zach, Christopher, Huu Le +1
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning and Algorithms #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2003.05886

openalex publication_date 2020/03/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Optimization problems with an auxiliary latent variable structure in addition to the main model parameters occur frequently in computer vision and machine learning. The additional latent variables make the underlying optimization task expensive, either in terms of memory (by maintaining the latent variables), or in terms of runtime (repeated exact inference of latent variables). We aim to remove the need to maintain the latent variables and propose two formally justified methods, that dynamically adapt the required accuracy of latent variable inference. These methods have applications in large scale robust estimation and in learning energy-based models from labeled data.

Citations

• On the Convergence Properties of the EM Algorithm
• Gradient-based learning applied to document recognition
• First-order methods of smooth convex optimization with inexact oracle
• A Tutorial on MM Algorithms
• Learning Multiple Layers of Features from Tiny Images

Related

• Learning with Latent Structures in Natural Language Processing: A Survey
• Leveraging Recursive Gumbel-Max Trick for Approximate Inference in Combinatorial Spaces
• Learning Multi-layer Latent Variable Model via Variational Optimization\n of Short Run MCMC for Approximate Inference
• Joint Stochastic Approximation and Its Application to Learning Discrete Latent Variable Models
• A Unified Theory of Exact Inference and Learning in Exponential Family Latent Variable Models