A family of statistical symmetric divergences based on Jensen's inequality

2010/09/21 by Frank Nielsen, Nielsen, Frank · 2 citations
Computer Science · Engineering · Physics and Astronomy · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Face and Expression Recognition #Information Theory (cs.IT) #Remote-Sensing Image Classification #Statistical Mechanics and Entropy

paper · pdf · doi:10.48550/arxiv.1009.4004

openalex publication_date 2010/09/21 · openalex created_date 2016/06/24 · openalex updated_date 2026/07/28

Abstract

We introduce a novel parametric family of symmetric information-theoretic distances based on Jensen's inequality for a convex functional generator. In particular, this family unifies the celebrated Jeffreys divergence with the Jensen-Shannon divergence when the Shannon entropy generator is chosen. We then design a generic algorithm to compute the unique centroid defined as the minimum average divergence. This yields a smooth family of centroids linking the Jeffreys to the Jensen-Shannon centroid. Finally, we report on our experimental results.

Citations

• On Information and Sufficiency
• Divergence measures based on the Shannon entropy

Cited by

• Automata Learning -- Expect Delays!
• Self-supervised Representation Learning with Relative Predictive Coding

Related

• A generalization of the Jensen divergence: The chord gap divergence
• Generalized Symmetric Divergence Measures and Inequalities
• Two tales for a geometric Jensen--Shannon divergence
• Bounds on Non-Symmetric Divergence Measures in Terms of Symmetric Divergence Measures
• Nonextensive Generalizations of the Jensen-Shannon Divergence