EP-GIG Priors and Applications in Bayesian Sparse Learning

2012/04/19 by Zhihua Zhang, Zhang, Zhihua, Shusen Wang +5
Computer Science · Mathematics · #Bayesian Methods and Mixture Models #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (stat.ML) #Statistical Methods and Inference

paper · pdf · doi:10.48550/arxiv.1204.4243

openalex publication_date 2012/04/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper we propose a novel framework for the construction of sparsity-inducing priors. In particular, we define such priors as a mixture of exponential power distributions with a generalized inverse Gaussian density (EP-GIG). EP-GIG is a variant of generalized hyperbolic distributions, and the special cases include Gaussian scale mixtures and Laplace scale mixtures. Furthermore, Laplace scale mixtures can subserve a Bayesian framework for sparse learning with nonconvex penalization. The densities of EP-GIG can be explicitly expressed. Moreover, the corresponding posterior distribution also follows a generalized inverse Gaussian distribution. These properties lead us to EM algorithms for Bayesian sparse learning. We show that these algorithms bear an interesting resemblance to iteratively re-weighted ℓ₂ or ℓ₁ methods. In addition, we present two extensions for grouped variable selection and logistic regression.

Citations

• The horseshoe estimator for sparse signals
• Iteratively reweighted least squares minimization for sparse recovery

Related

• Sparsity enforcing priors in inverse problems via Normal variance mixtures: model selection, algorithms and applications
• The Inverse Gamma-Gamma Prior for Optimal Posterior Contraction and Multiple Hypothesis Testing
• Rectified Gaussian Scale Mixtures and the Sparse Non-Negative Least Squares Problem
• Block Sparse Bayesian Learning: A Diversified Scheme
• Learning a Gaussian Mixture for Sparsity Regularization in Inverse Problems