From Kinetic Theory to AI: a Rediscovery of High-Dimensional Divergences and Their Properties

2025/07/15 by Gennaro Auricchio, Giovanni Brigati, Auricchio, Gennaro +5 · 4 citations
Computer Science · Materials Science · #35B40 #35K55 #35L60 #35Q70 #35Q91 #35Q92 #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #FOS: Physical sciences #Machine Learning (cs.LG) #Machine Learning in Materials Science #Mathematical Physics (math-ph) #Multiagent Systems (cs.MA) #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.2507.11387

openalex publication_date 2025/07/15 · openalex created_date 2025/10/08 · openalex updated_date 2026/08/01

Abstract

Selecting an appropriate divergence measure is a critical aspect of machine learning, as it directly impacts model performance. Among the most widely used, we find the Kullback-Leibler (KL) divergence, originally introduced in kinetic theory as a measure of relative entropy between probability distributions. Just as in machine learning, the ability to quantify the proximity of probability distributions plays a central role in kinetic theory. In this paper, we present a comparative review of divergence measures rooted in kinetic theory, highlighting their theoretical foundations and exploring their potential applications in machine learning and artificial intelligence.

Citations

• On the equivalence between static and dynamic optimal transport governed by linear control systems
• Kinetic Optimal Transport (OTIKIN) -- Part 1: Second-Order Discrepancies Between Probability Measures
• Hypocoercivity meets lifts
• Multivariate Gini-type discrepancies
• Extending the Gini Index to Higher Dimensions via Whitening Processes
• Explicit convergence rates of underdamped Langevin dynamics under weighted and weak Poincaré--Lions inequalities
• Benamou-Brenier Formulation of Optimal Transport for Nonlinear Control Systems on Rd
• Non-reversible lifts of reversible diffusion processes and relaxation times
• Measuring multidimensional inequality: a new proposal based on the Fourier transform
• Splitting Methods for differential equations
• How to construct decay rates for kinetic Fokker--Planck equations?
• The Fourier Discrepancy Function
• Fast and Smooth Interpolation on Wasserstein Space
• Computing Kantorovich-Wasserstein Distances on d-dimensional\n histograms using (d+1)-partite graphs
• Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problem
• Second order models for optimal transport and cubic splines on the Wasserstein space
• Measure-valued spline curves: an optimal transport viewpoint
• Wealth distribution in presence of debts. A Fokker--Planck description
• The Cramer Distance as a Solution to Biased Wasserstein Gradients
• Sparsifying the Fisher Linear Discriminant by Rotation
• Hypocoercivity for linear kinetic equations conserving mass
• Comparing Measures of Sparsity
• Statistical physics of social dynamics
• On the self-similar asymptotics for generalized non-linear kinetic Maxwell models
• Fourier-Based Distances and Berry-Esseen Like Inequalities for Smooth Densities
• Generalization of an Inequality by Talagrand and Links with the Logarithmic Sobolev Inequality
• Wealth condensation in a simple model of economy
• Fisher information estimates for Boltzmann's collision operator
• Polar factorization and monotone rearrangement of vector‐valued functions
• Mathematical Methods of Organizing and Planning Production
• Theory of Statistical Estimation
• The Measurement of the Inequality of Incomes
• A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem

Cited by

• On Rank Graduation Metrics for High Dimensional Ordinal Data
• Kinetic and mean-field modeling of muscular dystrophies
• Large-time behaviour for coupled systems of Lotka-Volterra-type Fokker-Planck equations
• Energy distance and evolution problems: a promising tool for kinetic equations

Related

• Notes on Kullback-Leibler Divergence and Likelihood
• Expansion of the Kullback-Leibler Divergence, and a new class of information metrics
• Analysis of Discriminator in RKHS Function Space for Kullback-Leibler Divergence Estimation
• A Neural Network Algorithm for KL Divergence Estimation with Quantitative Error Bounds
• On the Kullback-Leibler divergence between pairwise isotropic Gaussian-Markov random fields