Informational Divergence and Entropy Rate on Rooted Trees with Probabilities

2013/10/10 by Georg Böcherer, Böcherer, Georg, Rana Ali Amjad +1
Computer Science · Mathematics · #FOS: Computer and information sciences #Information Theory (cs.IT) #cs.IT #math.IT

paper · pdf · doi:10.48550/arxiv.1310.2882

5 pages. With proofs and illustrating example

arxiv created 2013/10/10 · arxiv updated 2013/10/11

Abstract

Rooted trees with probabilities are used to analyze properties of a variable length code. A bound is derived on the difference between the entropy rates of the code and a memoryless source. The bound is in terms of normalized informational divergence. The bound is used to derive converses for exact random number generation, resolution coding, and distribution matching.

Related

• Rooted Trees with Probabilities Revisited
• Comparing entropy rates on finite and infinite rooted trees
• On Strong Data-Processing and Majorization Inequalities with Applications to Coding Problems
• Fixed-to-Variable Length Resolution Coding for Target Distributions
• Relations between entropy and error probability