2021/05/19 by Yo Sheena, Sheena, Yo
Mathematics · #Advanced Statistical Methods and Models #FOS: Mathematics #Primary 60F99 #Secondary 62F12 #Statistical Distribution Estimation and Applications #Statistical Methods and Bayesian Inference #Statistics Theory (math.ST)
paper · pdf · doi:10.48550/arxiv.2105.08947
openalex publication_date 2021/05/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
For a parametric model of distributions, the closest distribution in the model to the true distribution located outside the model is considered. Measuring the closeness between two distributions with the Kullback-Leibler (K-L) divergence, the closest distribution is called the "information projection." The estimation risk of the maximum likelihood estimator (MLE) is defined as the expectation of K-L divergence between the information projection and the predictive distribution with plugged-in MLE. Here, the asymptotic expansion of the risk is derived up to n-2-order, and the sufficient condition on the risk for the Bayes error rate between the true distribution and the information projection to be lower than a specified value is investigated. Combining these results, the "p-n criterion" is proposed, which determines whether the MLE is sufficiently close to the information projection for the given model and sample. In particular, the criterion for an exponential family model is relatively simple and can be used for a complex model with no explicit form of normalizing constant. This criterion can constitute a solution to the sample size or model acceptance problem. Use of the p-n criteria is demonstrated for two practical datasets. The relationship between the results and information criteria is also studied.