Mutual Information second order expansion is the Pearson's chi-square statistic

2026/07/20 by Marius Marinescu
#math.ST #stat.TH

paper · pdf

Abstract

We show that MI connects subtly and elegantly the two best-known state-of-the-art independence test statistics: the G² and the Pearson's chi-square statistic χ². Furthermore, we show that the MI connects directly those statistics by an elegant formula arising from a stochastic Taylor expansion of MI (δ-method). MI second order term is precisely χ² up to a scale factor. As a consequence, by this connection, the difference between G² and χ² can be explicitly quantified.

Citations

• Measuring and testing dependence by correlation of distances

Related

• Mutual information of Contingency Tables and Related Inequalities
• On the use of Mutual Information for Testing Independence
• Seven proofs of the Pearson Chi-squared independence test and its\n graphical interpretation
• On the asymptotic distributions of some test statistics for two-way contingency tables
• Information Divergence is more chi squared distributed than the chi squared statistics