2020/11/01 by Yuhang Cai, Lek‐Heng Lim, Cai, Yuhang +1 · 1 citation
Mathematics · Physics and Astronomy · #28A33 #28A50 #46E27 #49Q22 #60E05 #94A17 #Advanced Statistical Methods and Models #FOS: Computer and information sciences #FOS: Mathematics #Information Theory (cs.IT) #Probability (math.PR) #Statistical Mechanics and Entropy #Statistics Theory (math.ST)
paper · pdf · doi:10.48550/arxiv.2011.00629
openalex publication_date 2020/11/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Comparing probability distributions is an indispensable and ubiquitous task in machine learning and statistics. The most common way to compare a pair of Borel probability measures is to compute a metric between them, and by far the most widely used notions of metric are the Wasserstein metric and the total variation metric. The next most common way is to compute a divergence between them, and in this case almost every known divergences such as those of Kullback--Leibler, Jensen--Shannon, Rényi, and many more, are special cases of the f-divergence. Nevertheless these metrics and divergences may only be computed, in fact, are only defined, when the pair of probability measures are on spaces of the same dimension. How would one quantify, say, a KL-divergence between the uniform distribution on the interval [-1,1] and a Gaussian distribution on ℝ3? We show that these common notions of metrics and divergences give rise to natural distances between Borel probability measures defined on spaces of different dimensions, e.g., one on ℝm and another on ℝn where m, n are distinct, so as to give a meaningful answer to the previous question.