2021/07/27 by Daniel R. Kowal, Kowal, Daniel R. · 1 citation
Computer Science · Mathematics · #Bayesian Modeling and Causal Inference #Computation (stat.CO) #FOS: Computer and information sciences #Machine Learning (stat.ML) #Methodology (stat.ME) #Statistical Methods and Bayesian Inference #Statistical Methods and Inference
paper · pdf · doi:10.48550/arxiv.2107.12890
openalex publication_date 2021/07/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Linear mixed models (LMMs) are instrumental for regression analysis with structured dependence, such as grouped, clustered, or multilevel data. However, selection among the covariates--while accounting for this structured dependence--remains a challenge. We introduce a Bayesian decision analysis for subset selection with LMMs. Using a Mahalanobis loss function that incorporates the structured dependence, we derive optimal linear coefficients for (i) any given subset of variables and (ii) all subsets of variables that satisfy a cardinality constraint. Crucially, these estimates inherit shrinkage or regularization and uncertainty quantification from the underlying Bayesian model, and apply for any well-specified Bayesian LMM. More broadly, our decision analysis strategy deemphasizes the role of a single "best" subset, which is often unstable and limited in its information content, and instead favors a collection of near-optimal subsets. This collection is summarized by key member subsets and variable-specific importance metrics. Customized subset search and out-of-sample approximation algorithms are provided for more scalable computing. These tools are applied to simulated data and a longitudinal physical activity dataset, and demonstrate excellent prediction, estimation, and selection ability.