2020/02/21 by Maciej Skorski, Maciej Skórski, Skorski, Maciej
Computer Science · Mathematics · Physics and Astronomy · #Gaussian Processes and Bayesian Inference #Markov Chains and Monte Carlo Methods #Statistical Mechanics and Entropy #cs.DS #cs.IT #math.IT #stat.CO
paper · pdf · doi:10.48550/arxiv.2002.09264
arxiv created 2020/02/21 · arxiv updated 2020/02/24
Entropy Estimation is an important problem with many applications in cryptography, statistic,machine learning. Although the estimators optimal with respect to the sample complexity have beenrecently developed, there are still some challenges we address in this paper.The contribution is a novel estimator which is built directly on the birthday paradox. Theanalysis turns out to be considerably simpler and offer superior confidence bounds with explicitconstants. We also discuss how streaming algorithm can be used to massively improve memoryconsumption. Last but not least, we study the problem of estimation in low or moderate regimes,adapting the estimator and proving rigorus bounds.