2022/12/20 by Lea Kaufmann, Kaufmann, Lea, Maria Kateri +1
Decision Sciences · Mathematics · #Advanced Statistical Methods and Models #Advanced Statistical Process Monitoring #Algorithm #Artificial intelligence #Categorical variable #Computer science #Consistency (knowledge bases) #Contrast (vision) #Coordinate descent #Covariate #Discrete mathematics #Elastic net regularization #FOS: Mathematics #Feature selection #Lasso (programming language) #Logistic regression #Mathematics #Metric (unit) #Oracle #Regression #Regularization (linguistics) #Selection (genetic algorithm) #Statistical Methods and Inference #Statistics #Statistics Theory (math.ST) #Variable (mathematics)
paper · pdf · doi:10.48550/arxiv.2212.10073
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2022/12/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Nowadays, several data analysis problems require for complexity reduction, mainly meaning that they target at removing the non-influential covariates from the model and at delivering a sparse model. When categorical covariates are present, with their levels being dummy coded, the number of parameters included in the model grows rapidly, fact that emphasizes the need for reducing the number of parameters to be estimated. In this case, beyond variable selection, sparsity is also achieved through fusion of levels of covariates which do not differentiate significantly in terms of their influence on the response variable. In this work a new regularization technique is introduced, called L0-Fused Group Lasso (L0-FGL) for binary logistic regression. It uses a group lasso penalty for factor selection and for the fusion part it applies an L0 penalty on the differences among the levels' parameters of a categorical predictor. Using adaptive weights, the adaptive version of L0-FGL method is derived. Theoretical properties, such as the existence, √(n) consistency and oracle properties under certain conditions, are established. In addition, it is shown that even in the diverging case where the number of parameters pn grows with the sample size n, √(n) consistency and a consistency in variable selection result are achieved. Two computational methods, PIRLS and a block coordinate descent (BCD) approach using quasi Newton, are developed and implemented. A simulation study supports that L0-FGL shows an outstanding performance, especially in the high dimensional case.