A closed-form expression for the Sharma-Mittal entropy of exponential families

2011/05/17 by Frank Nielsen, Richard Nock · 1 citation
Computer Science · Mathematics · Physics and Astronomy · #Entropy (arrow of time) #Exponential family #Exponential function #Expression (computer science) #Gaussian #Kullback–Leibler divergence #Natural exponential family #Random Matrices and Applications #Statistical Distribution Estimation and Applications #Statistical Mechanics and Entropy #cs.IT #cs.LG #math.IT

paper · pdf · doi:10.1088/1751-8113/45/3/032003

published as Journal of Physics A: Mathematical and Theoretical 45.3 (2011): 032003 · 9 pages, 3 figures; Journal of Physics A: Mathematical and Theoretical, December 2011. IOP

arxiv created 2011/12/19 · openalex publication_date 2011/12/19 · openalex created_date 2019/06/27 · arxiv updated 2021/04/29 · openalex updated_date 2026/08/05

Abstract

The Sharma-Mittal entropies generalize the celebrated Shannon, Rényi and Tsallis entropies. We report a closed-form formula for the Sharma-Mittal entropies and relative entropies for arbitrary exponential family distributions. We instantiate explicitly the formula for the case of the multivariate Gaussian distributions and discuss on its estimation.

Cited by

• Rényi entropy [wikipedia]

Related

• Is Sharma-Mittal entropy really a step beyond Tsallis and Renyi entropies?
• Entropies and cross-entropies of exponential families
• Properties of the Shannon, Rényi and other entropies: dependence in parameters, robustness in distributions and extremes
• Applications of Entropy in Data Analysis and Machine Learning: A Review
• Learn your entropy from informative data: an axiom ensuring the consistent identification of generalized entropies