2021/11/13 by Luc Devroye, Laszlo Gyorfi, László Györfi · 1 citation
Mathematics · #Mathematical Analysis and Transform Methods #Nonlinear Partial Differential Equations #Advanced Harmonic Analysis Research
paper · doi:10.1109/tit.2021.3127938
We revisit the problem of the estimation of the differential entropy <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">H(f) </tex-math></inline-formula> of a random vector <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">X </tex-math></inline-formula> in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">Rd </tex-math></inline-formula> with density <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">f </tex-math></inline-formula> , assuming that <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">H(f) </tex-math></inline-formula> exists and is finite. In this note, we study the consistency of the popular nearest neighbor estimate <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">Hn </tex-math></inline-formula> of Kozachenko and Leonenko. Without any smoothness condition we show that the estimate is consistent ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">E\|Hn - H(f)|\ → 0 </tex-math></inline-formula> as <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">n → ∞ </tex-math></inline-formula> ) if and only if <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">\mathbb E\ log (‖ X ‖ + 1)\ < ∞ </tex-math></inline-formula> . Furthermore, if <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">X </tex-math></inline-formula> has compact support, then <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">Hn → H(f) </tex-math></inline-formula> almost surely.