2006/01/18 by Ambedkar Dukkipati, Shalabh Bhatnagar, Dukkipati, Ambedkar +3
Economics, Econometrics and Finance · Physics and Astronomy · #Advanced Thermodynamics and Statistical Mechanics #Complex Systems and Time Series Analysis #FOS: Physical sciences #Mathematical Physics (math-ph) #Statistical Mechanics and Entropy
paper · pdf · doi:10.48550/arxiv.math-ph/0601035
openalex publication_date 2006/01/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The measure-theoretic definition of Kullback-Leibler relative-entropy (KL-entropy) plays a basic role in the definitions of classical information measures. Entropy, mutual information and conditional forms of entropy can be expressed in terms of KL-entropy and hence properties of their measure-theoretic analogs will follow from those of measure-theoretic KL-entropy. These measure-theoretic definitions are key to extending the ergodic theorems of information theory to non-discrete cases. A fundamental theorem in this respect is the Gelfand-Yaglom-Perez (GYP) Theorem (Pinsker, 1960, Theorem. 2.4.2) which states that measure-theoretic relative-entropy equals the supremum of relative-entropies over all measurable partitions. This paper states and proves the GYP-theorem for Renyi relative-entropy of order greater than one. Consequently, the result can be easily extended to Tsallis relative-entropy.