A New ECDF Two-Sample Test Statistic

2020/07/02 by Connor Dowd, Dowd, Connor · 1 citation
Mathematics · #FOS: Computer and information sciences #Methodology (stat.ME) #Other Statistics (stat.OT) #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #Statistical Methods in Clinical Trials

paper · pdf · doi:10.48550/arxiv.2007.01360

openalex publication_date 2020/07/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Empirical cumulative distribution functions (ECDFs) have been used to test the hypothesis that two samples come from the same distribution since the seminal contribution by Kolmogorov and Smirnov. This paper describes a statistic which is usable under the same conditions as Kolmogorov-Smirnov, but provides more power than other extant tests in that vein. I demonstrate a valid (conservative) procedure for producing finite-sample p-values. I outline the close relationship between this statistic and its two main predecessors. I also provide a public R package (CRAN: twosamples [2018]) implementing the testing procedure in O(Nlog(N)) time with O(N) memory. Using the package's functions, I perform several simulation studies showing the power improvements.

Citations

• Asymptotic Theory of Certain "Goodness of Fit" Criteria Based on Stochastic Processes
• Table for Estimating the Goodness of Fit of Empirical Distributions

Cited by

• kNNDM CV: k -fold nearest-neighbour distance matching cross-validation for map accuracy estimation

Related

• Numerically more stable computation of the p-values for the two-sample Kolmogorov-Smirnov test
• Two samples test for discrete power-law distributions
• The power of visualizing distributional differences: Formal graphical n-sample tests
• The Kolmogorov-Smirnov Statistic Revisited
• A novel statistical approach for two-sample testing based on the overlap coefficient