2014/02/01 by Peter Harremoës, Harremoës, Peter
Computer Science · Mathematics · #FOS: Computer and information sciences #FOS: Mathematics #Information Theory (cs.IT) #Statistics Theory (math.ST) #cs.IT #math.IT #math.ST #stat.TH
paper · pdf · doi:10.48550/arxiv.1402.0092
A version without the appendix has been submitted to a conference
arxiv created 2014/02/01 · arxiv updated 2014/02/04
For testing independence it is very popular to use either the χ2-statistic or G2-statistics (mutual information). Asymptotically both are χ2-distributed so an obvious question is which of the two statistics that has a distribution that is closest to the χ2-distribution. Surprisingly the distribution of mutual information is much better approximated by a χ2-distribution than the χ2-statistic. For technical reasons we shall focus on the simplest case with one degree of freedom. We introduce the signed log-likelihood and demonstrate that its distribution function can be related to the distribution function of a standard Gaussian by inequalities. For the hypergeometric distribution we formulate a general conjecture about how close the signed log-likelihood is to a standard Gaussian, and this conjecture gives much more accurate estimates of the tail probabilities of this type of distribution than previously published results. The conjecture has been proved numerically in all cases relevant for testing independence and further evidence of its validity is given.