Data augmentation for non-Gaussian regression models using variance-mean\n mixtures

2011/03/28 by Nicholas G. Polson, James G. Scott, Polson, Nicholas G. +1 · 1 citation
Chemistry · Computer Science · Engineering · Mathematics · #Advanced Statistical Methods and Models #Applied mathematics #Artificial intelligence #Bayesian probability #Computer science #Control Systems and Identification #Econometrics #Expectation–maximization algorithm #Gaussian #Gaussian Processes and Bayesian Inference #Logistic regression #Mathematics #Maximum likelihood #Prior probability #Quantile regression #Regression #Regression analysis #Regularization (linguistics) #Robustness (evolution) #Spectroscopy and Chemometric Analyses #Statistical Methods and Inference #Statistics #Variance (accounting) #stat.CO #stat.ME

paper · pdf · doi:10.48550/arxiv.1103.5407

published in arXiv (Cornell University) (Cornell University) · Added a discussion of quasi-Newton acceleration

openalex publication_date 2011/03/28 · arxiv created 2012/09/22 · arxiv updated 2012/09/25 · openalex created_date 2022/09/24 · openalex updated_date 2026/08/06

Abstract

We use the theory of normal variance-mean mixtures to derive a\ndata-augmentation scheme for a class of common regularization problems. This\ngeneralizes existing theory on normal variance mixtures for priors in\nregression and classification. It also allows variants of the\nexpectation-maximization algorithm to be brought to bear on a wider range of\nmodels than previously appreciated. We demonstrate the method on several\nexamples, including sparse quantile regression and binary logistic regression.\nWe also show that quasi-Newton acceleration can substantially improve the speed\nof the algorithm without compromising its robustness.\n

Citations

• Asymptotic properties of bridge estimators in sparse high-dimensional regression models
• Least angle regression
• The horseshoe estimator for sparse signals
• Large-Scale Bayesian Logistic Regression for Text Categorization
• Multinomial Inverse Regression for Text Analysis
• Local shrinkage rules, Levy processes, and regularized regression

Cited by

• Fast Compute for ML Optimization

Related

• On the Data Augmentation Algorithm for Bayesian Multivariate Linear Regression with Non-Gaussian Errors
• Mixtures of Gaussian process priors
• Gaussian Mixture Model with unknown diagonal covariances via continuous sparse regularization
• Self-regularizing Property of Nonparametric Maximum Likelihood Estimator in Mixture Models
• Lasso regularization for mixture experiments with noise variables