2020/10/24 by Jincheng Bai, Bai, Jincheng, Qifan Song +3
Computer Science · Mathematics · #FOS: Computer and information sciences #FOS: Mathematics #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #Statistics Theory (math.ST) #cs.LG #math.ST #stat.ML #stat.TH
paper · pdf · doi:10.48550/arxiv.2010.12887
arxiv created 2020/10/24 · openalex publication_date 2020/10/24 · arxiv updated 2020/10/27 · openalex created_date 2020/10/29 · openalex updated_date 2026/07/28
We propose a variational Bayesian (VB) procedure for high-dimensional linear model inferences with heavy tail shrinkage priors, such as student-t prior. Theoretically, we establish the consistency of the proposed VB method and prove that under the proper choice of prior specifications, the contraction rate of the VB posterior is nearly optimal. It justifies the validity of VB inference as an alternative of Markov Chain Monte Carlo (MCMC) sampling. Meanwhile, comparing to conventional MCMC methods, the VB procedure achieves much higher computational efficiency, which greatly alleviates the computing burden for modern machine learning applications such as massive data analysis. Through numerical studies, we demonstrate that the proposed VB method leads to shorter computing time, higher estimation accuracy, and lower variable selection error than competitive sparse Bayesian methods.