2010/09/10 by Guang Cheng, Cheng, Guang
Engineering · Mathematics · #Control Systems and Identification #FOS: Mathematics #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #Statistics Theory (math.ST)
paper · pdf · doi:10.48550/arxiv.1009.2111
openalex publication_date 2010/09/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
A common practice in obtaining a semiparametric efficient estimate is through iteratively maximizing the (penalized) log-likelihood w.r.t. its Euclidean parameter and functional nuisance parameter via Newton-Raphson algorithm. The purpose of this paper is to provide a formula in calculating the minimal number of iterations k^∗ needed to produce an efficient estimate θn(k^∗) from a theoretical point of view. We discover that (a) k^∗ depends on the convergence rates of the initial estimate and nuisance estimate; (b) more than k^∗ iterations, i.e., k, will only improve the higher order asymptotic efficiency of θn(k); (c) k^∗ iterations are also sufficient for recovering the estimation sparsity in high dimensional data. These general conclusions hold, in particular, when the nuisance parameter is not estimable at root-n rate, and apply to semiparametric models estimated under various regularizations, e.g., kernel or penalized estimation. This paper provides a first general theoretical justification for the "one-/two-step iteration" phenomena observed in the literature, and may be useful in reducing the bootstrap computational cost for the semiparametric models.