Law of log determinant of sample covariance matrix and optimal estimation of differential entropy for high-dimensional Gaussian distributions

2013/09/02 by T. Tony Cai, Tommaso Cai, Tengyuan Liang +1 · 69 citations
Computer Science · Economics, Econometrics and Finance · Mathematics · #Applied mathematics #Bayesian Methods and Mixture Models #Covariance #Covariance matrix #Differential entropy #Entropy (arrow of time) #Estimation of covariance matrices #Financial Risk and Volatility Modeling #Gaussian #Mathematics #Maximum entropy probability distribution #Principle of maximum entropy #Sample mean and sample covariance #Scatter matrix #Statistical Methods and Inference #Statistical physics #Statistics #cs.IT #math.IT #math.ST #stat.TH

paper · pdf · doi:10.1016/j.jmva.2015.02.003

published in Journal of Multivariate Analysis 137, 161-172 (Elsevier BV) · 19 pages

arxiv created 2013/09/02 · openalex publication_date 2015/02/20 · arxiv updated 2015/03/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06

Abstract

Differential entropy and log determinant of the covariance matrix of a multivariate Gaussian distribution have many applications in coding, communications, signal processing and statistical inference. In this paper we consider in the high dimensional setting optimal estimation of the differential entropy and the log-determinant of the covariance matrix. We first establish a central limit theorem for the log determinant of the sample covariance matrix in the high dimensional setting where the dimension p(n) can grow with the sample size n. An estimator of the differential entropy and the log determinant is then considered. Optimal rate of convergence is obtained. It is shown that in the case p(n)/n → 0 the estimator is asymptotically sharp minimax. The ultra-high dimensional setting where p(n) > n is also discussed.

Citations

• The Distribution of the Determinant of a Complex Wishart Distributed Matrix

Cited by

• Bernstein-von Mises Theorems for Functionals of Covariance Matrix
• Linear spectral statistics of sequential sample covariance matrices
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction
• Understanding Neural Networks with Logarithm Determinant Entropy Estimator
• Configurable photonic simulator for quantum field dynamics
• In-Context Occam's Razor: How Transformers Prefer Simpler Hypotheses on the Fly
• Spectral Estimation with Free Decompression
• BOLT: Block-Orthonormal Lanczos for Trace estimation of matrix functions
• Optimal Uncertainty Size in Distributionally Robust Inverse Covariance Estimation
• Mean-field theory of Bayesian clustering

Related

• High-dimensional covariance estimation by minimizing ℓ₁-penalized log-determinant divergence
• Entropic Determinants
• Log-minor distributions and an application to estimating mean subsystem entropy
• Estimation of a high-dimensional covariance matrix with the Stein loss
• Entropic Trace Estimates for Log Determinants