The Risk Inflation Criterion for Multiple Regression

1994/12/01 by Dean P. Foster, Edward I. George · 14 citations
Decision Sciences · Mathematics · #Advanced Statistical Methods and Models #Optimal Experimental Design Methods #Statistical Methods and Inference

paper · pdf · doi:10.1214/aos/1176325766

openalex publication_date 1994/12/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/22

Abstract

A new criterion is proposed for the evaluation of variable selection procedures in multiple regression. This criterion, which we call the risk inflation, is based on an adjustment to the risk. Essentially, the risk inflation is the maximum increase in risk due to selecting rather than knowing the "correct" predictors. A new variable selection procedure is obtained which, in the case of orthogonal predictors, substantially improves on AIC, C_p and BIC and is close to optimal. In contrast to AIC, C_p and BIC which use dimensionality penalties of 2, 2 and log n, respectively, this new procedure uses a penalty 2 log p, where p is the number of available predictors. For the case of nonorthogonal predictors, bounds for the optimal penalty are obtained.

Citations

• The Relationship Between Variable Selection and Data Agumentation and a Method for Prediction
• Ideal spatial adaptation by wavelet shrinkage

Cited by

• Model Selection
• Frequentist Model Average Estimators
• Focused Information Criteria and Model Averaging for the Cox Hazard Regression Model
• Sparse estimators and the oracle property, or the return of Hodges’ estimator
• Variable selection in semiparametric regression modeling
• The sparsity and bias of the Lasso selection in high-dimensional linear regression
• General empirical Bayes wavelet methods and exactly adaptive minimax estimation
• Extended BIC for small-n-large-P sparse GLM
• Model Selection Techniques -- An Overview
• Akaike's Information Criterion, Cp and Estimators of Loss for Elliptically Symmetric Distributions
• Can the strengths of AIC and BIC be shared? A conflict between model indentification and regression estimation
• Model selection and psychological theory: A discussion of the differences between the Akaike information criterion (AIC) and the Bayesian information criterion (BIC).
• General maximum likelihood empirical Bayes estimation of normal means
• Stepwise regression [wikipedia]