2014/02/16 by Shifeng Xiong, Xiong, Shifeng
Mathematics · #Advanced Statistical Methods and Models #Computation (stat.CO) #FOS: Computer and information sciences #FOS: Mathematics #Statistical Methods and Bayesian Inference #Statistics Education and Methodologies #Statistics Theory (math.ST)
paper · pdf · doi:10.48550/arxiv.1402.3748
openalex publication_date 2014/02/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Many statistical methods require solutions to optimization problems. When the global solution is hard to attain, statisticians always use the better if there are two solutions for chosen, where the word "better" is understood in the sense of optimization. This seems reasonable in that the better solution is more likely to be the global solution, whose statistical properties of interest usually have been well established. From the statistical perspective, we use the better solution because we intuitively believe the principle, called better solution principle (BSP) in this paper, that a better solution to a statistical optimization problem also has better statistical properties of interest. BSP displays some concordance between optimization and statistics, and is expected to widely hold. Since theoretical study on BSP seems to be neglected by statisticians, this paper aims to establish a framework for discussing BSP in various statistical optimization problems. We demonstrate several simple but effective comparison theorems as the key results of this paper, and apply them to verify BSP in commonly encountered statistical optimization problems, including maximum likelihood estimation, best subsample selection, and best subset regression. It can be seen that BSP for these problems holds under reasonable conditions, i.e., a better solution indeed has better statistical properties of interest. In addition, guided by the BSP theory, we develop a new best subsample selection method that performs well when there are clustered outliers.