A Variational View on Statistical Multiscale Estimation

2021/06/10 by Markus Haltmeier, Housen Li, Axel Munk · 4 citations
Engineering · Mathematics · #Applied mathematics #Artificial intelligence #Augmented Lagrangian method #Computer science #Context (archaeology) #Convex optimization #Duality (order theory) #Estimator #Inverse problem #Mathematical analysis #Mathematical optimization #Mathematics #Numerical methods in inverse problems #Regular polygon #Regularization (linguistics) #Sparse and Compressive Sensing Techniques #Statistical Methods and Inference #Statistics #math.ST #msc:62-02 #msc:62G05 #stat.CO #stat.ME #stat.TH

paper · pdf · open access · doi:10.1146/annurev-statistics-040120-030531

published in Annual Review of Statistics and Its Application 9(1), 343-372 (Annual Reviews)

arxiv created 2021/06/10 · openalex publication_date 2022/03/07 · arxiv updated 2022/03/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06

Abstract

We present a unifying view on various statistical estimation techniques including penalization, variational and thresholding methods. These estimators will be analyzed in the context of statistical linear inverse problems including nonparametric and change point regression, and high dimensional linear models as examples. Our approach reveals many seemingly unrelated estimation schemes as special instances of a general class of variational multiscale estimators, named MIND (MultIscale Nemirovskii--Dantzig). These estimators result from minimizing certain regularization functionals under convex constraints that can be seen as multiple statistical tests for local hypotheses. For computational purposes, we recast MIND in terms of simpler unconstraint optimization problems via Lagrangian penalization as well as Fenchel duality. Performance of several MINDs is demonstrated on numerical examples.

Citations

• A data-driven block thresholding approach to wavelet estimation
• The Relationship Between Variable Selection and Data Agumentation and a Method for Prediction
• Can one estimate the conditional distribution of post-model-selection estimators?
• Wavelet Shrinkage Denoising Using the Non-Negative Garrote
• Nonlinear total variation based noise removal algorithms
• De-noising by soft-thresholding
• Ideal spatial adaptation by wavelet shrinkage
• Smooth minimization of non-smooth functions
• Optimal rates for total variation denoising
• Empirical Risk Minimization as Parameter Choice Rule for General Linear\n Regularization Methods
• Calibrating the scan statistic: finite sample performance vs. asymptotics
• Variational Multiscale Nonparametric Regression: Algorithms and Implementation
• Frame-constrained Total Variation Regularization for White Noise\n Regression
• Better Subset Regression Using the Nonnegative Garrote