2020/04/06 by Luis Miguel Lopez-Ramos, Lopez-Ramos, Luis Miguel, Baltasar Beferull-Lozano +1
Computer Science · Engineering · Mathematics · #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Signal Processing (eess.SP) #cs.LG #eess.SP #electronic engineering #information engineering #stat.ML
paper · pdf · doi:10.48550/arxiv.2004.02769
6 pages, 3 figures, 1 algorithm; Submitted to the European Signal Processing Conference (EUSIPCO) 2020 (Amsterdam)
arxiv created 2020/04/06 · arxiv updated 2020/04/07
There is a clear need for efficient algorithms to tune hyperparameters for statistical learning schemes, since the commonly applied search methods (such as grid search with N-fold cross-validation) are inefficient and/or approximate. Previously existing algorithms that efficiently search for hyperparameters relying on the smoothness of the cost function cannot be applied in problems such as Lasso regression. In this contribution, we develop a hyperparameter optimization method that relies on the structure of proximal gradient methods and does not require a smooth cost function. Such a method is applied to Leave-one-out (LOO)-validated Lasso and Group Lasso to yield efficient, data-driven, hyperparameter optimization algorithms. Numerical experiments corroborate the convergence of the proposed method to a local optimum of the LOO validation error curve, and the efficiency of its approximations.