Statistical Inference in Classification of High-dimensional Gaussian Mixture

2024/10/25 by Hanwen Huang, Huang, Hanwen, Peng Zeng +1
Computer Science · #Statistical and Computational Modeling

paper · pdf · doi:10.48550/arxiv.2410.19950

Abstract

We consider the classification problem of a high-dimensional mixture of two Gaussians with general covariance matrices. Using the replica method from statistical physics, we investigate the asymptotic behavior of a general class of regularized convex classifiers in the high-dimensional limit, where both the sample size n and the dimension p approach infinity while their ratio α=n/p remains fixed. Our focus is on the generalization error and variable selection properties of the estimators. Specifically, based on the distributional limit of the classifier, we construct a de-biased estimator to perform variable selection through an appropriate hypothesis testing procedure. Using L₁-regularized logistic regression as an example, we conducted extensive computational experiments to confirm that our analytical findings are consistent with numerical simulations in finite-sized systems. We also explore the influence of the covariance structure on the performance of the de-biased estimator.

Related

• High Dimensional Classification via Regularized and Unregularized Empirical Risk Minimization: Precise Error and Optimal Loss
• A Large Dimensional Study of Regularized Discriminant Analysis Classifiers
• Interpolating Discriminant Functions in High-Dimensional Gaussian Latent Mixtures
• Classifying Overlapping Gaussian Mixtures in High Dimensions: From Optimal Classifiers to Neural Nets
• Binary Classification of Gaussian Mixtures: Abundance of Support Vectors, Benign Overfitting and Regularization