2014/03/10 by Li Ma, Ma, Li
Computer Science · Mathematics · #62F15 #62J99 #62M05 #Bayesian Methods and Mixture Models #Computation (stat.CO) #FOS: Computer and information sciences #Methodology (stat.ME) #Statistical Methods and Bayesian Inference #Statistical Methods and Inference
paper · pdf · doi:10.48550/arxiv.1403.2397
openalex publication_date 2014/03/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We show that a probabilistic version of the classical forward-stepwise variable inclusion procedure can serve as a general data-augmentation scheme for model space distributions in (generalized) linear models. This latent variable representation takes the form of a Markov process, thereby allowing information propagation algorithms to be applied for sampling from model space posteriors. In particular, we propose a sequential Monte Carlo method for achieving effective unbiased Bayesian model averaging in high-dimensional problems, utilizing proposal distributions constructed using local information propagation. We illustrate our method---called LIPS for local information propagation based sampling---through real and simulated examples with dimensionality ranging from 15 to 1,000, and compare its performance in estimating posterior inclusion probabilities and in out-of-sample prediction to those of several other methods---namely, MCMC, BAS, iBMA, and LASSO. In addition, we show that the latent variable representation can also serve as a modeling tool for specifying model space priors that account for knowledge regarding model complexity and conditional inclusion relationships.