2011/05/04 by Georg Tauboeck, Georg Tauböck
Computer Science · Mathematics · #Binary entropy function #Cellular Automata and Applications #Channel capacity #Coding (social sciences) #Computability, Logic, AI Algorithms #Computer science #Entropy (arrow of time) #Entropy power inequality #Entropy rate #Information theory #Mathematical Dynamics and Fractals #Mathematics #Physics #Principle of maximum entropy #Statistical physics #Statistics #cs.IT #math.IT
paper · pdf · doi:10.1109/tit.2012.2184638
33 pages, 1 figure, slightly modified version of first paper revision submitted to IEEE Trans. Inf. Theory on October 31, 2010
arxiv created 2011/05/04 · openalex publication_date 2012/01/31 · openalex created_date 2016/06/24 · arxiv updated 2016/11/17 · openalex updated_date 2026/08/05
Recent research has demonstrated significant achievable performance gains by exploiting circularity/noncircularity or properness/improperness of complex-valued signals. In this paper, we investigate the influence of these properties on important information theoretic quantities such as entropy, divergence, and capacity. We prove two maximum entropy theorems that strengthen previously known results. The proof of the first maximum entropy theorem is based on the so-called circular analog of a given complex-valued random vector. The introduction of the circular analog is additionally supported by a characterization theorem that employs a minimum Kullback-Leibler divergence criterion. In the proof of the second maximum entropy theorem, results about the second-order structure of complex-valued random vectors are exploited. Furthermore, we address the capacity of multiple-input multiple-output (MIMO) channels. Regardless of the specific distribution of the channel parameters (noise vector and channel matrix, if modeled as random), we show that the capacity-achieving input vector is circular for a broad range of MIMO channels (including coherent and noncoherent scenarios). Finally, we investigate the situation of an improper and Gaussian distributed noise vector. We compute both capacity and capacity-achieving input vector and show that improperness increases capacity, provided that the complementary covariance matrix is exploited. Otherwise, a capacity loss occurs, for which we derive an explicit expression.