Model Selection Techniques -- An Overview

2018/10/22 by Jie Ding, Vahid Tarokh, Yuhong Yang · 4 citations
Computer Science · Economics, Econometrics and Finance · Engineering · Mathematics · Physics and Astronomy · #Advanced Statistical Methods and Models #Anomaly Detection Techniques and Applications #Artificial intelligence #Big data #Computer science #Data mining #Data science #Engineering #Epistemology #Inference #Machine learning #Management science #Mathematics #Model selection #Predictive power #Selection (genetic algorithm) #Set (abstract data type) #Statistical Methods and Inference #Statistical inference #Statistical model #Statistics #cs.IT #cs.LG #econ.EM #math.IT #physics.app-ph #stat.ML

paper · pdf · doi:10.1109/msp.2018.2867638

accepted by IEEE SIGNAL PROCESSING MAGAZINE

arxiv created 2018/10/22 · arxiv updated 2018/10/24 · openalex publication_date 2018/11/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

In the era of big data, analysts usually explore various statistical models or machine learning methods for observed data in order to facilitate scientific discoveries or gain predictive power. Whatever data and fitting procedures are employed, a crucial step is to select the most appropriate model or method from a set of candidates. Model selection is a key ingredient in data analysis for reliable and reproducible statistical inference or prediction, and thus central to scientific studies in fields such as ecology, economics, engineering, finance, political science, biology, and epidemiology. There has been a long history of model selection techniques that arise from researches in statistics, information theory, and signal processing. A considerable number of methods have been proposed, following different philosophies and exhibiting varying performances. The purpose of this article is to bring a comprehensive overview of them, in terms of their motivation, large sample performance, and applicability. We provide integrated and practically relevant discussions on theoretical properties of state-of- the-art model selection approaches. We also share our thoughts on some controversial views on the practice of model selection.

Citations

• An Asymptotic Equivalence of Choice of Model by Cross-Validation and Akaike's Criterion
• Nearly unbiased variable selection under minimax concave penalty
• The Risk Inflation Criterion for Multiple Regression
• Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter
• Sparsest solutions of underdetermined linear systems via ℓq-minimization for 0<q⩽1
• Sparse estimators and the oracle property, or the return of Hodges’ estimator
• The Large-Sample Distribution of the Likelihood Ratio for Testing Composite Hypotheses
• On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities
• The Relationship Between Variable Selection and Data Agumentation and a Method for Prediction
• Accumulated prediction errors, information criteria and optimal forecasting for autoregressive time series
• Least angle regression
• Order selection for same-realization predictions in autoregressive processes
• Akaike's Information Criterion in Generalized Estimating Equations
• Gradient-based learning applied to document recognition
• A new look at the statistical model identification
• Why Most Published Research Findings Are False
• Stability Selection
• Statistical Learning with Sparsity: The Lasso and Generalizations
• Present Position and Potential Developments: Some Personal Views: Statistical Theory: The Prequential Approach
• SPARLS: The Sparse RLS Algorithm
• On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities

Cited by

• Regression model selection via log‐likelihood ratio and constrained minimum criterion
• No Cross-Validation Required: An Analytical Framework for Regularized Mixed-Integer Problems (Extended Version)
• Akaike information criterion [wikipedia]
• Model selection [wikipedia]

Related

• Editorial: Model Selection and Efficiency—is ‘Which Model …?’ the Right Question?
• Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning
• Morris Hansen Lecture 2023: Model Selection and Its Important Roles in Surveys
• Statistical Model Choice
• A Predictive Approach to Model Selection