High-dimensional regression with outcomes of mixed-type using the multivariate spike-and-slab LASSO

2025/06/16 by Ghosh, Soham, Deshpande, Sameer K. · 1 citation
#FOS: Computer and information sciences #Methodology (stat.ME)

paper · doi:10.48550/arxiv.2506.13007

Abstract

We consider a high-dimensional multi-outcome regression in which q, possibly dependent, binary and continuous outcomes are regressed onto p covariates. We model the observed outcome vector as a partially observed latent realization from a multivariate linear regression model. Our goal is to estimate simultaneously a sparse matrix (B) of latent regression coefficients (i.e., partial covariate effects) and a sparse latent residual precision matrix (Ω), which induces partial correlations between the observed outcomes. To this end, we specify continuous spike-and-slab priors on all entries of B and off-diagonal elements of Ω and introduce a Monte Carlo Expectation-Conditional Maximization algorithm to compute the maximum a posterior estimate of the model parameters. Under a set of mild assumptions, we derive the posterior contraction rate for our model in the high-dimensional regimes where both p and q diverge with the sample size n and establish a sure screening property, which implies that, as n increases, we can recover all truly non-zero elements of B with probability tending to one. We demonstrate the excellent finite-sample properties of our proposed method, which we call mixed-mSSL, using extensive simulation studies and three applications spanning medicine to ecology.

Citations

• Compressive Bayesian non-negative matrix factorization for mutational signatures analysis
• Posterior consistency in multi-response regression models with non-informative priors for the error covariance matrix in growing dimensions
• Bayesian Sparse Regression for Mixed Multi-Responses with Application to Runtime Metrics Prediction in Fog Manufacturing
• Posterior contraction and uncertainty quantification for the multivariate spike-and-slab LASSO
• Estimating sparse direct effects in multivariate regression with the spike-and-slab LASSO
• Two-Step Mixed-Type Multivariate Bayesian Sparse Variable Selection with Shrinkage Priors
• Dimension-free Mixing for High-dimensional Bayesian Variable Selection
• Contraction of a quasi-Bayesian model with shrinkage priors in precision matrix estimation
• Precision Matrix Estimation under the Horseshoe-like Prior-Penalty Dual
• Mixed-type multivariate response regression with covariance estimation
• Bayesian Bootstrap Spike-and-Slab LASSO
• Integrals over Gaussians under Linear Domain Constraints
• Simultaneous Transformation and Rounding (STAR) Models for\n Integer-Valued Data
• Nearly optimal Bayesian Shrinkage for High Dimensional Regression
• Metagenomic analysis of faecal microbiome as a tool towards targeted non-invasive biomarkers for colorectal cancer
• The Horseshoe+ Estimator of Ultra-Sparse Signals
• Bayesian inference for logistic models using Polya-Gamma latent\n variables
• Bayesian Inference for Logistic Models Using Pólya–Gamma Latent Variables
• Bayesian multivariate mixed-scale density estimation

Cited by

• Scalable Bayesian inference for high-dimensional mixed-type multivariate spatial data

Related

• Posterior contraction and uncertainty quantification for the multivariate spike-and-slab LASSO
• Estimating sparse direct effects in multivariate regression with the spike-and-slab LASSO
• Estimation, Confidence Intervals, and Large-Scale Hypotheses Testing for High-Dimensional Mixed Linear Regression
• Bayesian Estimation of Sparse Spiked Covariance Matrices in High Dimensions
• Bayesian Linear Regression for Multivariate Responses Under Group Sparsity