Entropy and the Discrete Central Limit Theorem

2021/06/01 by Gavalakis, Lampros, Kontoyiannis, Ioannis · 4 citations
#60E15 #60F05 #94A17 #FOS: Computer and information sciences #FOS: Mathematics #Information Theory (cs.IT) #Probability (math.PR)

paper · doi:10.48550/arxiv.2106.00514

Abstract

A strengthened version of the central limit theorem for discrete random variables is established, relying only on information-theoretic tools and elementary arguments. It is shown that the relative entropy between the standardised sum of n independent and identically distributed lattice random variables and an appropriately discretised Gaussian, vanishes as n→∞.

Cited by

• Improved Bounds for the Freiman-Ruzsa Theorem
• The LLN and CLT for the statistical ensembles of discrete integrable Hamiltonian systems
• Conditions for equality and stability in Shannon's and Tao's entropy power inequalities
• The entropic coherence is a necessary resource for non-energy preserving gates

Related

• Maximum entropy approach to central limit distributions of correlated variables
• On the entropy growth of sums of iid discrete random variables
• Entropies and their Asymptotic Theory in the discrete case
• Convergence to Stable Laws in Relative Entropy
• Sharp Concentration Inequalities for the Centered Relative Entropy