Information Complexity Criterion for Model Selection in Robust Regression Using A New Robust Penalty Term

2020/12/04 by Pamukçu, Esra, Çankaya, Mehmet Niyazi
#62B05 #62F10 #62J05 #FOS: Mathematics #Statistics Theory (math.ST)

paper · doi:10.48550/arxiv.2012.02468

Abstract

Model selection is basically a process of finding the best model from the subset of models in which the explanatory variables are effective on the response variable. The log likelihood function for the lack of fit term and a specified penalty term are used as two parts in a model selection criteria. In this paper, we derive a new tool for the model selection in robust regression. We introduce a new definition of relative entropy based on objective functions. Due to the analytical simplicity, we use Huber's objective function ρ_H and propose our specified penalty term C₀^ρ_H to derive new Information Complexity Criterion (RICOMP_C0^ρH) as a robust model selection tool. Additionally, by using the properties of C₀^ρ_H, we propose a new value of tuning parameter called k_C0 for the Huber's ρ_H. If a contamination to normal distribution exists, RICOMP_C0^ρH chooses the true model better than the rival ones. Monte Carlo Simulation studies are carried out to show the utility both of k_C0 and RICOMP_C0^ρH. A real data example is also given.

Related

• Model Selection for independent not identically distributed observations based on Rényi's pseudodistances
• Kernel-based Information Criterion
• Robust Variable Selection Criteria for the Penalized Regression
• Robust model selection in generalized linear models
• Model Selection and Akaike's Information Criterion (AIC): The General Theory and its Analytical Extensions