2022/05/20 by Mohammad-Amin Charusaie, Arash Amini, Stefano Rini · 1 citation
Engineering · Mathematics · #Sparse and Compressive Sensing Techniques #Mathematical Analysis and Transform Methods #Wireless Communication Security Techniques #Differential entropy #Mathematics #Random variable #Entropy (arrow of time) #Probability distribution #Lipschitz continuity #Discrete mathematics #Applied mathematics #Combinatorics #Pure mathematics #Principle of maximum entropy #Binary entropy function #Statistics #Physics
paper · doi:10.1109/tit.2022.3174623
openalex publication_date 2022/05/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29
The notion of compressibility of a random measure is a rather general concept which find applications in many contexts from data compression, to signal quantization, and parameter estimation. While compressibility for discrete and continuous measures is generally well understood, the case of discrete-continuous measures is quite subtle. In this paper, we focus on a class of multi-dimensional random measures that have singularities on affine lower-dimensional subsets. We refer to this class of random variables asaffinely singular. Affinely singular random vectors naturally arises when considering linear transformation of component-wise independent discrete-continuous random variables. To measure the compressibility of such distributions, we introduce the new notion of dimensional-rate bias (DRB) which is closely related to the entropy and differential entropy in discrete and continuous cases, respectively. Similar to entropy and differential entropy, DRB is useful in evaluating the mutual information between distributions of the aforementioned type. Besides the DRB, we also evaluate the the RID of these distributions. We further provide an upper-bound for the RID of multi-dimensional random measures that are obtained by Lipschitz functions of component-wise independent discrete-continuous random variables (X). The upper-bound is shown to be achievable when the Lipschitz function isA \mathrm X, whereAsatisfies SPARK(Am× n) = m+1(e.g., Vandermonde matrices). When considering discrete-domain moving-average processes with non-Gaussian excitation noise, the above results allow us to evaluate the block-average RID and DRB, as well as to determine a relationship between these parameters and other existing compressibility measures.