Penalty, Shrinkage, and Preliminary Test Estimators under Full Model\n Hypothesis

2015/03/24 by Enayetur Raheem, Raheem, Enayetur, A. K. Ehsanes Saleh +1
Mathematics · #Advanced Statistical Methods and Models #Computation (stat.CO) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (stat.ML) #Methodology (stat.ME) #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #Statistics Theory (math.ST)

paper · pdf · doi:10.48550/arxiv.1503.06910

openalex publication_date 2015/03/24 · openalex created_date 2022/08/29 · openalex updated_date 2026/07/28

Abstract

This paper considers a multiple regression model and compares, under full\nmodel hypothesis, analytically as well as by simulation, the performance\ncharacteristics of some popular penalty estimators such as ridge regression,\nLASSO, adaptive LASSO, SCAD, and elastic net versus Least Squares Estimator,\nrestricted estimator, preliminary test estimator, and Stein-type estimators\nwhen the dimension of the parameter space is smaller than the sample space\ndimension. We find that RR uniformly dominates LSE, RE, PTE, SE and PRSE while\nLASSO, aLASSO, SCAD, and EN uniformly dominates LSE only. Further, it is\nobserved that neither penalty estimators nor Stein-type estimator dominate one\nanother.\n

Citations

• Rejoinder to "Least angle regression" by Efron et al
• On the adaptive elastic-net with a diverging number of parameters
• A Statistical View of Some Chemometrics Regression Tools

Related

• Restricted LASSO and Double Shrinking
• Lasso and equivalent quadratic penalized models
• Positive-shrinkage and Pretest Estimation in Multiple Regression: A Monte Carlo study with Applications
• Testing Sparsity-Inducing Penalties
• Degrees of Freedom in Penalized Regression: Model Selection with Adaptive Penalties