2024/06/02 by Neri Merhav, Merhav, Neri, Nir Weinberger +1 · 1 citation
Computer Science · #FOS: Computer and information sciences #Information Theory (cs.IT) #Neural Networks and Applications
paper · pdf · doi:10.48550/arxiv.2406.00744
openalex publication_date 2024/06/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
This monograph offers a toolbox of mathematical techniques, which have been effective and widely applicable in information-theoretic analysis. The first tool is a generalization of the method of types to Gaussian settings, and then to general exponential families. The second tool is Laplace and saddle-point integration, which allow to refine the results of the method of types, and are capable of obtaining more precise results. The third is the type class enumeration method, a principled method to evaluate the exact random-coding exponent of coded systems, which results in the best known exponent in various problem settings. The fourth subset of tools aimed at evaluating the expectation of non-linear functions of random variables, either via integral representations, or by a refinement of Jensen's inequality via change-of-measure, by complementing Jensen's inequality with a reversed inequality, or by a class of generalized Jensen's inequalities that are applicable for functions beyond convex/concave. Various application examples of all these tools are provided along this monograph.