On Bayesian robust regression with diverging number of predictors

2015/07/08 by Daniel Nevo, Nevo, Daniel, Ya'acov Ritov +1
Mathematics · #FOS: Computer and information sciences #FOS: Mathematics #Methodology (stat.ME) #Statistics Theory (math.ST) #math.ST #stat.ME #stat.TH

paper · pdf · doi:10.48550/arxiv.1507.02074

18 pages

arxiv created 2016/04/04 · arxiv updated 2016/04/06

Abstract

This paper concerns the robust regression model when the number of predictors and the number of observations grow in a similar rate. Theory for M-estimators in this regime has been recently developed by several authors [El Karoui et al., 2013, Bean et al., 2013, Donoho and Montanari, 2013]. Motivated by the inability of M-estimators to successfully estimate the Euclidean norm of the coefficient vector, we consider a Bayesian framework for this model. We suggest a two-component mixture of normals prior for the coefficients and develop a Gibbs sampler procedure for sampling from relevant posterior distributions, while utilizing a scale mixture of normal representation for the error distribution . Unlike M-estimators, the proposed Bayes estimator is consistent in the Euclidean norm sense. Simulation results demonstrate the superiority of the Bayes estimator over traditional estimation methods.

Related

• Reconciling Bayesian and frequentist approaches to robustness against outliers
• A General Method for Robust Bayesian Modeling
• A Bayesian “Sandwich” for Variance Estimation
• Bayesian Model Robustness via Disparities
• Bayesian shrinkage prediction for the regression problem