Regression model selection via log‐likelihood ratio and constrained minimum criterion

2023/01/10 by Min Tsao · 1 citation
Mathematics · #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #Advanced Statistical Methods and Models #Akaike information criterion #Bayesian information criterion #Likelihood-ratio test #Statistics #Deviance information criterion #Model selection #Mathematics #Frequentist inference #Information Criteria #Likelihood principle #Sample size determination #Score test #Regression analysis #Selection (genetic algorithm) #Ratio test #Bayesian probability #Likelihood function #Maximum likelihood #Bayesian inference #Computer science #Artificial intelligence #Quasi-maximum likelihood

paper · pdf · doi:10.1002/cjs.11756

openalex publication_date 2023/01/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/06/02

Abstract

Abstract Although log‐likelihood is widely used in model selection, the log‐likelihood ratio has had few applications in this area. We develop a log‐likelihood ratio based method for selecting regression models by focusing on the set of models deemed plausible by the likelihood ratio test. We show that when the sample size is large and the significance level of the test is small, there is a high probability that the smallest model in this set is the true model; thus, we select this smallest model. The significance level of the test serves as a tuning parameter of this method. We consider three levels of this parameter in a simulation study and compare this method with the Akaike information criterion (AIC) and Bayesian information criterion (BIC) to demonstrate its excellent accuracy and adaptability to different sample sizes. This method is a frequentist alternative and a strong competitor to AIC and BIC for selecting regression models.

Citations

• Regression and time series model selection in small samples
• Application of Model-Selection Criteria to Some Problems in Multivariate Analysis
• Asymptotic properties of the maximum likelihood estimator in dichotomous logit models
• Consistency and Asymptotic Normality of the Maximum Likelihood Estimator in Generalized Linear Models
• A new look at the statistical model identification
• The Elements of Statistical Learning: Data Mining, Inference, and Prediction
• Model Selection Techniques -- An Overview
• Bridging AIC and BIC: A New Criterion for Autoregression

Cited by

• Model selection [wikipedia]

Related

• Regression model selection via log-likelihood ratio and constrained minimum criterion
• Linear regression model selection using p-values when the model dimension grows
• Model selection in logistic regression
• Selecting the Best Linear Regression Model: A Classical Approach
• Model Selection Confidence Sets by Likelihood Ratio Testing