2015/04/30 by Tobias Koch
Computer Science · Engineering · Mathematics · #Advanced Data Compression Techniques #Bandwidth (computing) #Coding (social sciences) #Combinatorics #Computer science #Differential entropy #Discrete mathematics #Distortion (music) #Entropy (arrow of time) #Finite set #Integer (computer science) #Mathematical analysis #Mathematics #Maximum entropy probability distribution #Physics #Principle of maximum entropy #Quantum mechanics #Rate distortion #Rate–distortion theory #Sparse and Compressive Sensing Techniques #Statistics #Telecommunications #Upper and lower bounds #Wireless Communication Security Techniques #cs.IT #math.IT
paper · pdf · doi:10.1109/tit.2016.2604254
13 pages, no figures. Replaced with version that has been submitted to IEEE Transactions on Information Theory. Lemma 1 has been generalized
arxiv created 2016/02/22 · openalex publication_date 2016/08/30 · arxiv updated 2016/11/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
The Shannon lower bound is one of the few lower bounds on the rate-distortion function that holds for a large class of sources. In this paper, which considers exclusively norm-based difference distortion measures, it is demonstrated that its gap to the rate-distortion function vanishes as the allowed distortion tends to zero for all sources having finite differential entropy and whose integer part has finite entropy. Conversely, it is demonstrated that if the integer part of the source has infinite entropy, then its rate-distortion function is infinite for every finite distortion level. Thus, the Shannon lower bound provides an asymptotically tight bound on the rate-distortion function if, and only if, the integer part of the source has finite entropy.