Geometric sensitivity of random matrix results: consequences for shrinkage estimators of covariance and related statistical methods

2011/05/07 by Noureddine El Karoui, Karoui, Noureddine El, Holger Koesters +1 · 5 citations
Computer Science · Mathematics · #Bayesian Methods and Mixture Models #FOS: Mathematics #Random Matrices and Applications #Statistical Methods and Inference #Statistics Theory (math.ST) #math.ST #stat.TH

paper · pdf · doi:10.48550/arxiv.1105.1404

68 pages

arxiv created 2011/05/07 · openalex publication_date 2011/05/07 · arxiv updated 2011/05/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Shrinkage estimators of covariance are an important tool in modern applied and theoretical statistics. They play a key role in regularized estimation problems, such as ridge regression (aka Tykhonov regularization), regularized discriminant analysis and a variety of optimization problems. In this paper, we bring to bear the tools of random matrix theory to understand their behavior, and in particular, that of quadratic forms involving inverses of those estimators, which are important in practice. We use very mild assumptions compared to the usual assumptions made in random matrix theory, requiring only mild conditions on the moments of linear and quadratic forms in our random vectors. In particular, we show that our results apply for instance to log-normal data, which are of interest in financial applications. Our study highlights the relative sensitivity of random matrix results (and their practical consequences) to geometric assumptions which are often implicitly made by random matrix theorists and may not be relevant in data analytic practice.

Citations

• A Distribution-Free M-Estimator of Multivariate Scatter
• On the Empirical Distribution of Eigenvalues of a Class of Large Dimensional Random Matrices
• A well-conditioned estimator for large-dimensional covariance matrices

Cited by

• Regularized Fingerprinting with Linearly Optimal Weight Matrix in Detection and Attribution of Climate Change
• An adaptable generalization of Hotelling's T² test in high dimension
• Asymptotics of Ridge (less) Regression under General Source Condition
• Graph connection Laplacian and random matrices with random blocks
• Ridge Regression: Structure, Cross-Validation, and Sketching

Related

• A shrinkage estimation for large dimensional precision matrices using random matrix theory
• Random matrix theory and robust covariance matrix estimation for financial data
• Improved Covariance Matrix Estimator using Shrinkage Transformation and Random Matrix Theory
• Matrix Means and a Novel High-Dimensional Shrinkage Phenomenon
• The lower tail of random quadratic forms, with applications to ordinary least squares and restricted eigenvalue properties