2018/01/31 by Yuta Sakai
Computer Science · Mathematics · Medicine · #Applied mathematics #Calculus (dental) #Combinatorics #Conditional mutual information #Fano plane #Inequality #Information theory #Limits and Structures in Graph Theory #Majorization #Markov Chains and Monte Carlo Methods #Mathematical Approximation and Integration #Mathematical analysis #Mathematical economics #Mathematics #Medicine #Mutual information #Physics #Pure mathematics #Statistical physics #Statistics #cs.IT #math.IT
paper · pdf · doi:10.3390/e22030288
published as Entropy, Volume 22, Issue 3, Paper 288, Year 2020. Available at https://www.mdpi.com/1099-4300/22/3/288 · 44 pages, 3 figures
openalex publication_date 2020/03/01 · arxiv created 2020/08/01 · arxiv updated 2020/08/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Fano’s inequality is one of the most elementary, ubiquitous, and important tools in information theory. Using majorization theory, Fano’s inequality is generalized to a broad class of information measures, which contains those of Shannon and Rényi. When specialized to these measures, it recovers and generalizes the classical inequalities. Key to the derivation is the construction of an appropriate conditional distribution inducing a desired marginal distribution on a countably infinite alphabet. The construction is based on the infinite-dimensional version of Birkhoff’s theorem proven by Révész [Acta Math. Hungar. 1962, 3, 188–198], and the constraint of maintaining a desired marginal distribution is similar to coupling in probability theory. Using our Fano-type inequalities for Shannon’s and Rényi’s information measures, we also investigate the asymptotic behavior of the sequence of Shannon’s and Rényi’s equivocations when the error probabilities vanish. This asymptotic behavior provides a novel characterization of the asymptotic equipartition property (AEP) via Fano’s inequality.