Wasserstein KL-divergence for Gaussian distributions

2025/03/31 by Datar, Adwait, Ay, Nihat
#FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (stat.ML) #Statistics Theory (math.ST)

paper · doi:10.48550/arxiv.2503.24022

Abstract

We introduce a new version of the KL-divergence for Gaussian distributions which is based on Wasserstein geometry and referred to as WKL-divergence. We show that this version is consistent with the geometry of the sample space \Bbb Rⁿ. In particular, we can evaluate the WKL-divergence of the Dirac measures concentrated in two points which turns out to be proportional to the squared distance between these points.

Related

• Limit laws of the empirical Wasserstein distance: Gaussian distributions
• Distances between probability distributions of different dimensions
• Statistical and Geometrical properties of regularized Kernel Kullback-Leibler divergence
• Independent Gaussian Distributions Minimize the Kullback-Leibler (KL) Divergence from Independent Gaussian Distributions
• Limit distribution theory for smooth p-Wasserstein distances