Finite-Sample Maximum Likelihood Estimation of Location

2022/06/06 by Gupta, Shivam, Lee, Jasper C. H., Price, Eric +1 · 1 citation
#Data Structures and Algorithms (cs.DS) #FOS: Computer and information sciences #FOS: Mathematics #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Statistics Theory (math.ST)

paper · doi:10.48550/arxiv.2206.02348

Abstract

We consider 1-dimensional location estimation, where we estimate a parameter λ from n samples λ+ η_i, with each η_i drawn i.i.d. from a known distribution f. For fixed f the maximum-likelihood estimate (MLE) is well-known to be optimal in the limit as n → ∞: it is asymptotically normal with variance matching the Cramér-Rao lower bound of (1)/(nI), where I is the Fisher information of f. However, this bound does not hold for finite n, or when f varies with n. We show for arbitrary f and n that one can recover a similar theory based on the Fisher information of a smoothed version of f, where the smoothing radius decays with n.

Cited by

• Near-Efficient and Non-Asymptotic Multiway Inference

Related

• High-dimensional Location Estimation via Norm Concentration for Subgamma Vectors
• Nonparametric Estimation of the Fisher Information and Its Applications
• On the Fisher Information of Discretized Data
• On improved estimation of the larger location parameter
• Maximum leave-one-out likelihood estimation for location parameter of unbounded densities