2020/10/20 by Patrick Schulte, Rana Ali Amjad, Schulte, Patrick +5
Computer Science · Mathematics · #FOS: Computer and information sciences #Information Theory (cs.IT) #cs.IT #math.IT
paper · pdf · doi:10.48550/arxiv.2010.10583
13 pages, 6 figures
arxiv created 2020/10/20 · arxiv updated 2020/10/22
Several applications in communication, control, and learning require approximating target distributions to within small informational divergence (I-divergence). The additional requirement of invertibility usually leads to using encoders that are one-to-one mappings, also known as distribution matchers. However, even the best one-to-one encoders have I-divergences that grow logarithmically with the block length in general. To improve performance, an encoder is proposed that has an invertible one-to-many mapping and a low-rate resolution code. Two algorithms are developed to design the mapping by assigning strings in either a most-likely first or least-likely first order. Both algorithms give information rates approaching the entropy of the target distribution with exponentially decreasing I-divergence and with vanishing resolution rate in the block length.