Covariance regularization by thresholding

2008/12/01 by Peter J. Bickel, Elizaveta Levina · 15 citations
Mathematics · Computer Science · Engineering · #Statistical Methods and Inference #Stochastic Gradient Optimization Techniques #Sparse and Compressive Sensing Techniques

paper · doi:10.1214/08-aos600

Abstract

This paper considers regularizing a covariance matrix of p variables estimated from n observations, by hard thresholding. We show that the thresholded estimate is consistent in the operator norm as long as the true covariance matrix is sparse in a suitable sense, the variables are Gaussian or sub-Gaussian, and (log p)/n→0, and obtain explicit rates. The results are uniform over families of covariance matrices which satisfy a fairly natural notion of sparsity. We discuss an intuitive resampling scheme for threshold selection and prove a general cross-validation result that justifies this approach. We also compare thresholding to other covariance estimators in simulations and on an example from climate data.

Citations

• On the distribution of the largest eigenvalue in principal components analysis
• Operator norm consistent estimation of large-dimensional sparse covariance matrices
• A well-conditioned estimator for large-dimensional covariance matrices
• Regularized estimation of large covariance matrices
• Ideal spatial adaptation by wavelet shrinkage

Cited by

• Restricted nonlinear shrinkage of high-dimensional residual covariance matrices in multivariate regressions
• Precise sample covariance spectral norm error -- an RDT view
• Sufficient Dimension Reduction via Inverse Conditional Mean or Variance Independence
• Robust and Well-conditioned Sparse Estimation for High-dimensional Covariance Matrices
• High dimensional matrix estimation through elliptical factor models
• Portfolio Optimization via Transfer Learning
• The entrywise calculus and dimension-free positivity preservers, with an Appendix on sphere packings
• Empirical Bayesian Multi-Bandit Learning
• An alternative bootstrap procedure for factor-augmented regression models
• Regularized estimation in sparse high-dimensional time series models
• Entrywise preservers of sign regularity
• Numerical study of high-dimensional covariance estimation and localization for data assimilation
• Bias Correction in Factor-Augmented Regression Models with Weak Factors
• Sparsity of the Main Effect Matrix Factor Model
• Tensor Elliptical Graphic Model

Related

• Regularized estimation of large covariance matrices
• Covariance Operator Estimation: Sparsity, Lengthscale, and Ensemble Kalman Filters
• Adaptive Thresholding for Sparse Covariance Matrix Estimation
• Covariance Operator Estimation via Adaptive Thresholding
• Sparse covariance thresholding for high-dimensional variable selection