Information Bottleneck on General Alphabets

2018/01/03 by Pichler, Georg, Koliander, Günther
#FOS: Computer and information sciences #Information Theory (cs.IT)

paper · doi:10.48550/arxiv.1801.01050

Abstract

We prove rigorously a source coding theorem that can probably be considered folklore, a generalization to arbitrary alphabets of a problem motivated by the Information Bottleneck method. For general random variables (Y, X), we show essentially that for some n ∈ ℕ, a function f with rate limit log|f| ≤ nR and I(Yⁿ; f(Xⁿ)) ≥ nS exists if and only if there is a random variable U such that the Markov chain Y - X - U holds, I(U; X) ≤ R and I(U; Y) ≥ S. The proof relies on the well established discrete case and showcases a technique for lifting discrete coding theorems to arbitrary alphabets.

Related

• Beyond Countable Alphabets: An Extension of the Information-Spectrum Approach
• Generalized Samorodnitsky noisy function inequalities, with applications to error-correcting codes
• Maximum Entropy Rate of Markov Sources for Systems With Non-regular Constraints
• Strong Converses using Typical Changes of Measures and Asymptotic Markov Chains
• Markov Lemma for Countable Alphabets