Asymptotic and finite-sample distributions of one- and two-sample empirical relative entropy

2025/12/18 by Matthieu Garcin, Louis Perot, Garcin, Matthieu +1
Economics, Econometrics and Finance · Mathematics · #stat.ME #math.ST #q-fin.ST #q-fin.TR #stat.AP #stat.TH

paper · pdf · doi:10.48550/arxiv.2512.16411

Abstract

In the perspective of building statistical tests of divergence between two probability distributions, we study the distribution of empirical relative entropy and derive several types of approximations: concentration inequalities for finite samples, asymptotic distributions, and Berry-Esseen bounds in a pre-asymptotic regime. For the latter, we introduce a new approach to obtain Berry-Esseen inequalities for nonlinear functions of sum statistics under some convexity assumptions. Our theoretical contributions cover both one- and two-sample empirical relative entropies.

Related

• Finite-sample concentration of the empirical relative entropy around its mean
• Sharp Concentration Inequalities for the Centered Relative Entropy
• Berry-Esseen bounds in the entropic central limit theorem
• Relative entropy bounds for sampling with and without replacement
• Entropy Concentration and the Empirical Coding Game