Chi-square and classical exact tests often wildly misreport significance; the remedy lies in computers

2011/08/20 by Perkins, William, Tygert, Mark, Ward, Rachel
#Computation (stat.CO) #FOS: Computer and information sciences #FOS: Mathematics #Methodology (stat.ME) #Statistics Theory (math.ST)

paper · doi:10.48550/arxiv.1108.4126

Abstract

If a discrete probability distribution in a model being tested for goodness-of-fit is not close to uniform, then forming the Pearson chi-square statistic can involve division by nearly zero. This often leads to serious trouble in practice -- even in the absence of round-off errors -- as the present article illustrates via numerous examples. Fortunately, with the now widespread availability of computers, avoiding all the trouble is simple and easy: without the problematic division by nearly zero, the actual values taken by goodness-of-fit statistics are not humanly interpretable, but black-box computer programs can rapidly calculate their precise significance.

Related

• Small-Sample Comparisons of Exact Levels for Chi-Squared Goodness-of-Fit Statistics
• Improving Accuracy of Goodness-of-fit Test
• Critical issues with the Pearson's chi-square test
• Zero-disparity Distribution Synthesis: Fast Exact Calculation of Chi-Squared Statistic Distribution for Discrete Uniform Histograms
• A Test of Goodness of Fit