Tight Lower Bounds for α-Divergences Under Moment Constraints and Relations Between Different α

2021/05/27 by Nishiyama, Tomohiro
#FOS: Computer and information sciences #FOS: Mathematics #Information Theory (cs.IT) #Statistics Theory (math.ST)

paper · doi:10.48550/arxiv.2105.12972

Abstract

The α-divergences include the well-known Kullback-Leibler divergence, Hellinger distance and χ²-divergence. In this paper, we derive differential and integral relations between the α-divergences that are generalizations of the relation between the Kullback-Leibler divergence and the χ²-divergence. We also show tight lower bounds for the α-divergences under given means and variances. In particular, we show a necessary and sufficient condition such that the binary divergences, which are divergences between probability measures on the same 2-point set, always attain lower bounds. Kullback-Leibler divergence, Hellinger distance, and χ²-divergence satisfy this condition.

Related

• A Tight Lower Bound for the Hellinger Distance with Given Means and Variances
• A New Lower Bound for Kullback-Leibler Divergence Based on Hammersley-Chapman-Robbins Bound
• Generalized Bregman and Jensen divergences which include some f-divergences
• Sequences of Inequalities Among New Divergence Measures
• Generalized Symmetric Divergence Measures and Inequalities