Exact Good-Turing characterization of the two-parameter Poisson-Dirichlet superpopulation model

2019/01/28 by Annalisa Cerquetti, Cerquetti, Annalisa
Computer Science · Mathematics · Biochemistry, Genetics and Molecular Biology · #Bayesian Methods and Mixture Models #Stochastic processes and statistical mechanics #Diffusion and Search Dynamics

paper · pdf · doi:10.48550/arxiv.1901.09665

Abstract

Large sample size equivalence between the celebrated \it approximated Good-Turing estimator of the probability to discover a species already observed a certain number of times (Good, 1953) and the modern Bayesian nonparametric counterpart has been recently established by virtue of a particular smoothing rule based on the two-parameter Poisson-Dirichlet model. Here we improve on this result showing that, for any finite sample size, when the population frequencies are assumed to be selected from a superpopulation with two-parameter Poisson-Dirichlet distribution, then Bayesian nonparametric estimation of the discovery probabilities corresponds to Good-Turing \it exact estimation. Moreover under general superpopulation hypothesis the Good-Turing solution admits an interpretation as a modern Bayesian nonparametric estimator under partial information.

Related

• Rediscovery of Good-Turing estimators via Bayesian nonparametrics
• A note on a Bayesian nonparametric estimator of the discovery probability
• On some Bayesian nonparametric estimators for species richness under two-parameter Poisson-Dirichlet priors
• Nonparametric Bayesian inference for the Gini-Simpson index
• Besting Good--Turing: Optimality of Non-Parametric Maximum Likelihood for Distribution Estimation