A systematic optimization approach for a class of statistical inference problems utilizing data augmentation

2016/01/19 by Rodrigo Carvajal, Rafael Orellana, Carvajal, Rodrigo +8
Computer Science · Decision Sciences · Engineering · Mathematics · #Advanced Multi-Objective Optimization Algorithms #Control Systems and Identification #FOS: Mathematics #Optimization and Control (math.OC) #Probabilistic and Robust Engineering Design #math.OC

paper · pdf · doi:10.48550/arxiv.1601.05074

18 pages, 3 figures

openalex publication_date 2016/01/19 · arxiv created 2018/05/18 · arxiv updated 2018/05/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present an algorithm for a class of statistical inference problems. The main idea is to reformulate the inference problem as an optimization procedure, based on the generation of surrogate (auxiliary) functions. This approach is motivated by the MM algorithm, combined with the systematic and iterative structure of the Expectation-Maximization algorithm. The resulting algorithm can deal with hidden variables in Maximum Likelihood and Maximum a Posteriori estimation problems, Instrumental Variables, Regularized Optimization and Constrained Optimization problems. The advantage of the proposed algorithm is to provide a systematic procedure to build surrogate functions for a class of problems where hidden variables are usually involved. Numerical examples show the benefits of the proposed approach.

Citations

• On scale mixtures of normal distributions
• A polyhedral branch-and-cut approach to global optimization
• Normal/Independent Distributions and Their Applications in Robust Regression
• A Tutorial on MM Algorithms
• Normal Variance-Mean Mixtures and z Distributions

Related

• A Tutorial on MM Algorithms
• Inference in a class of optimization problems: Confidence regions and finite sample bounds on errors in coverage probabilities
• Subsampled Optimization: Statistical Guarantees, Mean Squared Error Approximation, and Sampling Method
• Approximate selective inference via maximum likelihood
• Local optimization-based statistical inference