2019/02/14 by Yury Polyanskiy, Yihong Wu, Polyanskiy, Yury +1
Computer Science · Mathematics · #62C20 #62G05 #62G07 #62G86 #Advanced Statistical Methods and Models #Bayesian Methods and Mixture Models #FOS: Computer and information sciences #FOS: Mathematics #Information Theory (cs.IT) #Statistical Methods and Inference #Statistics Theory (math.ST)
paper · pdf · doi:10.48550/arxiv.1902.05616
openalex publication_date 2019/02/14 · openalex created_date 2021/01/18 · openalex updated_date 2026/07/28
Le Cam's method (or the two-point method) is a commonly used tool for obtaining statistical lower bound and especially popular for functional estimation problems. This work aims to explain and give conditions for the tightness of Le Cam's lower bound in functional estimation from the perspective of convex duality. Under a variety of settings it is shown that the maximization problem that searches for the best two-point lower bound, upon dualizing, becomes a minimization problem that optimizes the bias-variance tradeoff among a family of estimators. For estimating linear functionals of a distribution our work strengthens prior results of Donoho-Liu \citeDL91 (for quadratic loss) by dropping the Hölderian assumption on the modulus of continuity. For exponential families our results extend those of Juditsky-Nemirovski \citeJN09 by characterizing the minimax risk for the quadratic loss under weaker assumptions on the exponential family. We also provide an extension to the high-dimensional setting for estimating separable functionals. Notably, coupled with tools from complex analysis, this method is particularly effective for characterizing the ``elbow effect'' -- the phase transition from parametric to nonparametric rates. As the main application we derive sharp minimax rates in the Distinct elements problem (given a fraction p of colored balls from an urn containing d balls, the optimal error of estimating the number of distinct colors is Θ(d^-(1)/(2)min\(p)/(1-p),1\)) and the Fisher's species problem (given n iid observations from an unknown distribution, the optimal prediction error of the number of unseen symbols in the next (unobserved) r ⋅ n observations is Θ(n^-min\(1)/(r+1),(1)/(2)\)).