Penalized regression with multiple loss functions and selection by vote

2020/06/29 by Guorong Dai, Dai, Guorong, Ursula U. Müller +1
Engineering · Mathematics · #Advanced Statistical Methods and Models #Control Systems and Identification #FOS: Computer and information sciences #Methodology (stat.ME) #Statistical Methods and Inference

paper · pdf · doi:10.48550/arxiv.2006.16361

openalex publication_date 2020/06/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This article considers a linear model in a high dimensional data scenario. We propose a process which uses multiple loss functions both to select relevant predictors and to estimate parameters, and study its asymptotic properties. Variable selection is conducted by a procedure called "vote", which aggregates results from penalized loss functions. Using multiple objective functions separately simplifies algorithms and allows parallel computing, which is convenient and fast. As a special example we consider a quantile regression model, which optimally combines multiple quantile levels. We show that the resulting estimators for the parameter vector are asymptotically efficient. Simulations and a data application confirm the three main advantages of our approach: (a) reducing the false discovery rate of variable selection; (b) improving the quality of parameter estimation; (c) increasing the efficiency of computation.

Related

• Variable selection in high-dimensional linear model with possibly asymmetric or heavy-tailed errors
• Nearly unbiased variable selection under minimax concave penalty
• Penalized Variable Selection in Multi-Parameter Regression Survival Modelling
• Variable Selection in High Dimensional Linear Regressions with Parameter Instability
• Variable Selection with Second-Generation P-Values