Better Subset Regression Using the Nonnegative Garrote

1995/11/01 by Leo Breiman · 3 citations
Mathematics · Engineering · #Advanced Statistical Methods and Models #Statistical Methods and Inference #Control Systems and Identification

paper · doi:10.1080/00401706.1995.10484371

Abstract

A new method, called the nonnegative (nn) garrote, is proposed for doing subset regression. It both shrinks and zeroes coefficients. In tests on real and simulated data, it produces lower prediction error than ordinary subset selection. It is also compared to ridge regression. If the regression equations generated by a procedure do not change drastically with small changes in the data, the procedure is called stable. Subset selection is unstable, ridge is very stable, and the nn-garrote is intermediate. Simulation results illustrate the effects of instability on prediction error.

Citations

• Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter
• A Statistical View of Some Chemometrics Regression Tools

Cited by

• Least angle regression
• Wavelet Shrinkage Denoising Using the Non-Negative Garrote
• Sample size for binary logistic prediction models: Beyond events per variable criteria

Related

• Exhuming nonnegative garrote from oblivion using suitable initial estimates- illustration in low and high-dimensional real data
• Automated Analysis of Experiments using Hierarchical Garrote
• Wavelet Shrinkage Denoising Using the Non-Negative Garrote
• Best Subset Selection in Reduced Rank Regression
• Non-null Shrinkage Regression and Subset Selection via the Fractional Ridge Regression