On The Sparse Bayesian Learning Of Linear Models

2015/02/11 by Yves Atchade, Atchade, Yves, Chia Chye Yee +1
Mathematics · Physics and Astronomy · #FOS: Computer and information sciences #Methodology (stat.ME) #Statistical Mechanics and Entropy #Statistical Methods and Bayesian Inference #Statistical Methods and Inference

paper · pdf · doi:10.48550/arxiv.1502.03416

openalex publication_date 2015/02/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This work is a re-examination of the sparse Bayesian learning (SBL) of linear regression models of Tipping (2001) in a high-dimensional setting. We propose a hard-thresholded version of the SBL estimator that achieves, for orthogonal design matrices, the non-asymptotic estimation error rate of σ√(slog p)/√(n), where n is the sample size, p the number of regressors, σ is the regression model standard deviation, and s the number of non-zero regression coefficients. We also establish that with high-probability the estimator identifies the non-zero regression coefficients. In our simulations we found that sparse Bayesian learning regression performs better than lasso (Tibshirani (1996)) when the signal to be recovered is strong.

Related

• A Bayesian Lasso based Sparse Learning Model
• Sparse Bayesian Lasso via a Variable-Coefficient ℓ₁ Penalty
• Hyperparameter Estimation for Sparse Bayesian Learning Models
• Bayesian High-dimensional Semi-parametric Inference beyond sub-Gaussian Errors
• Inference in high-dimensional regression models without the exact or L^p sparsity