Data augmentation for non-Gaussian regression models using variance-mean\n mixtures

2011/03/28 by Nicholas G. Polson, James G. Scott, Polson, Nicholas G. +1
Computer Science · Chemistry · Engineering · #Gaussian Processes and Bayesian Inference #Spectroscopy and Chemometric Analyses #Control Systems and Identification

paper · pdf · doi:10.48550/arxiv.1103.5407

Abstract

We use the theory of normal variance-mean mixtures to derive a\ndata-augmentation scheme for a class of common regularization problems. This\ngeneralizes existing theory on normal variance mixtures for priors in\nregression and classification. It also allows variants of the\nexpectation-maximization algorithm to be brought to bear on a wider range of\nmodels than previously appreciated. We demonstrate the method on several\nexamples, including sparse quantile regression and binary logistic regression.\nWe also show that quasi-Newton acceleration can substantially improve the speed\nof the algorithm without compromising its robustness.\n

Citations

• Asymptotic properties of bridge estimators in sparse high-dimensional regression models
• Least angle regression
• The horseshoe estimator for sparse signals
• Large-Scale Bayesian Logistic Regression for Text Categorization
• Multinomial Inverse Regression for Text Analysis
• Local shrinkage rules, Levy processes, and regularized regression

Related

• On the Data Augmentation Algorithm for Bayesian Multivariate Linear Regression with Non-Gaussian Errors
• Mixtures of Gaussian process priors
• Gaussian Mixture Model with unknown diagonal covariances via continuous sparse regularization
• Self-regularizing Property of Nonparametric Maximum Likelihood Estimator in Mixture Models
• Lasso regularization for mixture experiments with noise variables