A provably convergent alternating minimization method for mean field\n inference

2015/02/20 by Pierre Baqué, Baqué, Pierre, Jean-Hubert Hours +5
Computer Science · Mathematics · #Bayesian Methods and Mixture Models #Statistical Methods and Inference #Statistical Methods and Bayesian Inference

paper · pdf · doi:10.48550/arxiv.1502.05832

Abstract

Mean-Field is an efficient way to approximate a posterior distribution in\ncomplex graphical models and constitutes the most popular class of Bayesian\nvariational approximation methods. In most applications, the mean field\ndistribution parameters are computed using an alternate coordinate\nminimization. However, the convergence properties of this algorithm remain\nunclear. In this paper, we show how, by adding an appropriate penalization\nterm, we can guarantee convergence to a critical point, while keeping a closed\nform update at each step. A convergence rate estimate can also be derived based\non recent results in non-convex optimization.\n

Citations

• Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization
• Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods
• On search directions for minimization algorithms

Related

• Principled Parallel Mean-Field Inference for Discrete Random Fields
• A Generalized Mean Field Algorithm for Variational Inference in Exponential Families
• Mean-field variational inference with the TAP free energy: Geometric and statistical properties in linear models
• Dissecting Non-Vacuous Generalization Bounds based on the Mean-Field\n Approximation
• Statistical Inference in Mean-Field Variational Bayes