Regression model selection via log-likelihood ratio and constrained minimum criterion

2021/07/18 by Min Tsao, Tsao, Min
Computer Science · Mathematics · #62J99 #Bayesian Methods and Mixture Models #FOS: Computer and information sciences #FOS: Mathematics #Methodology (stat.ME) #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #Statistics Theory (math.ST)

paper · pdf · doi:10.48550/arxiv.2107.08529

openalex publication_date 2021/07/18 · openalex created_date 2021/08/02 · openalex updated_date 2026/07/28

Abstract

Although the log-likelihood is widely used in model selection, the log-likelihood ratio has had few applications in this area. We develop a log-likelihood ratio based method for selecting regression models by focusing on the set of models deemed plausible by the likelihood ratio test. We show that when the sample size is large and the significance level of the test is small, there is a high probability that the smallest model in the set is the true model; thus, we select this smallest model. The significance level of the test serves as a parameter for this method. We consider three levels of this parameter in a simulation study and compare this method with the Akaike Information Criterion and Bayesian Information Criterion to demonstrate its excellent accuracy and adaptability to different sample sizes. We also apply this method to select a logistic regression model for a South African heart disease dataset.

Citations

• Application of Model-Selection Criteria to Some Problems in Multivariate Analysis
• Consistency and Asymptotic Normality of the Maximum Likelihood Estimator in Generalized Linear Models

Related

• Model selection in logistic regression
• Linear regression model selection using p-values when the model dimension grows
• A Robust Consistent Information Criterion for Model Selection based on Empirical Likelihood
• Selecting the Best Linear Regression Model: A Classical Approach
• Model Selection Confidence Sets by Likelihood Ratio Testing