Efficient sampling for sparse Bayesian learning using hierarchical prior normalization

2025/05/29 by Glaubitz, Jan, Marzouk, Youssef · 3 citations
#FOS: Mathematics #Numerical Analysis (math.NA)

paper · doi:10.48550/arxiv.2505.23753

Abstract

We introduce an approach for efficient Markov chain Monte Carlo (MCMC) sampling for challenging high-dimensional distributions in sparse Bayesian learning (SBL). The core innovation involves using hierarchical prior-normalizing transport maps (TMs), which are deterministic couplings that transform the sparsity-promoting SBL prior into a standard normal one. We analytically derive these prior-normalizing TMs by leveraging the product-like form of SBL priors and Knothe--Rosenblatt (KR) rearrangements. These transform the complex target posterior into a simpler reference distribution equipped with a standard normal prior that can be sampled more efficiently. Specifically, one can leverage the standard normal prior by using more efficient, structure-exploiting samplers. Our numerical experiments on various inverse problems -- including signal deblurring, inverting the non-linear inviscid Burgers equation, and recovering an impulse image -- demonstrate significant performance improvements for standard MCMC techniques.

Citations

• Priorconditioned Sparsity-Promoting Projection Methods for Deterministic and Bayesian Linear Inverse Problems
• Continuous Gaussian mixture solution for linear Bayesian inversion with application to Laplace priors
• A First Course in Monte Carlo Methods
• Scalability of Metropolis-within-Gibbs schemes for high-dimensional Bayesian models
• Hierarchical Bayesian Inverse Problems: A High-Dimensional Statistics Viewpoint
• Computationally efficient sampling methods for sparsity promoting hierarchical Bayesian models
• Path-following methods for Maximum a Posteriori estimators in Bayesian hierarchical models: How estimates depend on hyperparameters
• Transport Elliptical Slice Sampling
• Horseshoe priors for edge-preserving linear Bayesian inversion
• Cauchy Markov Random Field Priors for Bayesian Inversion
• A hybrid Gibbs sampler for edge-preserving tomographic reconstruction with uncertain view angles
• Sparsity promoting hybrid solvers for hierarchical Bayesian inverse problems
• Hyperparameter Estimation in Bayesian MAP Estimation: Parameterizations\n and Consistency
• Optimal Scaling of Random-Walk Metropolis Algorithms on General Target Distributions
• Rank-Normalization, Folding, and Localization: An Improved Rˆ for Assessing Convergence of MCMC (with Discussion)
• Certified dimension reduction in nonlinear Bayesian inverse problems
• Bayesian inverse problems with l₁ priors: a Randomize-then-Optimize approach
• Cauchy difference priors for edge-preserving Bayesian inversion with an\n application to X-ray tomography
• Fast sampling in a linear-Gaussian inverse problem
• Variational Inference with Normalizing Flows
• Julia: A Fresh Approach to Numerical Computing
• Julia: A Fresh Approach to Numerical Computing
• Algorithms for Kullback-Leibler Approximation of Probability Measures in\n Infinite Dimensions
• Analysis of the Gibbs sampler for hierarchical inverse problems
• Optimal tuning of the Hybrid Monte-Carlo Algorithm
• Elliptical slice sampling
• The Bayesian Lasso
• A General Framework for the Parametrization of Hierarchical Models
• General Methods for Monitoring Convergence of Iterative Simulations
• Inference from Iterative Simulation Using Multiple Sequences
• Remarks on a Multivariate Transformation
• Contributions to the theory of convex bodies.

Cited by

• Do Generalized-Gamma Scale Mixtures of Normals Fit Large Image Datasets?
• Joint Signal Recovery and Uncertainty Quantification via the Residual Prior Transform
• The Bayesian SIAC filter

Related

• Computationally efficient sampling methods for sparsity promoting hierarchical Bayesian models
• Learning Informed Prior Distributions with Normalizing Flows for Bayesian Analysis
• Efficient Bayesian Sampling Using Normalizing Flows to Assist Markov Chain Monte Carlo Methods
• Cost free hyper-parameter selection/averaging for Bayesian inverse problems with vanilla and Rao-Blackwellized SMC Samplers
• Sparsity enforcing priors in inverse problems via Normal variance mixtures: model selection, algorithms and applications