Estimating eigenvectors and eigenspaces of covariance matrices: Optimal Bounds and Conditions for Consistency

2026/07/27 by Phuc Tran, Van Vu
#math.ST #cs.NA #math.NA #math.PR #stat.TH

paper · pdf

Abstract

Let X = [ ξ₁, ξ₂,... ,ξ_d]^\top be a zero-mean random vector of large dimension d (d → ∞) with (hidden) covariance matrix M = (m_ij)_{1 ≤ i, j ≤ d}, where m_ij = m_ji = Cov(ξ_i, ξ_j). Let X₁, X₂, …, X_n be n iid samples of X. Consider the sample covariance matrix \textstyle M := (1)/(n) ∑_i=1ⁿ X_i X_i^\top. In practice, one frequently uses the eigenvectors and eigenspaces of M as estimators for those of M. A central task is to provide an error analysis for these estimators. In this paper, we provide an optimal error analysis, obtaining upper and lower bounds of matching order of magnitude, for a wide range of parameters d and n, under mild assumptions on M. As corollaries, we obtain new necessary and sufficient conditions for the consistency of the estimators. In these conditions, we only require the number of samples n to depend linearly on the effective rank of M, which can be much smaller than the dimension d.

Citations

• New perturbation bounds for low rank approximation of matrices: Beyond Eckart-Young-Mirsky
• Perturbation Bounds for Low-Rank Inverse Approximations under Noise
• Spectral Perturbation Bounds for Low-Rank Approximation with Applications to Privacy
• Davis-Kahan Theorem under a moderate gap condition
• Fast exact recovery of noisy matrix from few entries: the infinity norm approach
• New matrix perturbation bounds via combinatorial expansion I: Perturbation of eigenspaces
• Dimension-free Bounds for Sums of Independent Matrices and Simple Tensors via the Variational Principle
• High-dimensional principal component analysis with heterogeneous missingness
• Perturbation bounds for eigenspaces under a relative gap condition
• Matrices with Gaussian noise: optimal estimates for singular subspace perturbation
• Relative perturbation bounds with applications to empirical covariance operators
• The two-to-infinity norm and singular subspace geometry with\n applications to high-dimensional statistics
• Sample covariance matrices of heavy-tailed distributions
• Perturbation of linear forms of singular vectors under Gaussian noise
• An Introduction to Matrix Concentration Inequalities
• Asymptotics and Concentration Bounds for Bilinear Forms of Spectral Projectors of Sample Covariance
• Concentration Inequalities and Moment Bounds for Sample Covariance Operators
• On the principal components of sample covariance matrices
• Random perturbation of low rank matrices: Improving classical bounds
• Hanson-Wright inequality and sub-gaussian concentration
• Surprising Asymptotic Conical Structure in Critical Sample Eigen-Directions
• Second order accurate distributed eigenvector computation for extremely\n large matrices
• Eigenvalues of Large Sample Covariance Matrices of Spiked Population Models
• On the distribution of the largest eigenvalue in principal components analysis
• Asymptotic theory for the principal component analysis of a vector random function: Some applications to statistical inference
• Analysis of a complex of statistical variables into principal components.

Related

• Operator norm consistent estimation of large-dimensional sparse covariance matrices
• An ℓ_∞ Eigenvector Perturbation Bound and Its Application to Robust Covariance Estimation
• Spectrum estimation for large dimensional covariance matrices using random matrix theory
• Eigenvectors of some large sample covariance matrix ensembles
• Direct Estimation of Eigenvalues of Large Dimensional Precision Matrix