2020/11/05 by Matthias J. Ehrhardt, Ehrhardt, Matthias J., Lindon Roberts +1
Computer Science · #FOS: Computer and information sciences #FOS: Mathematics #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Machine Learning and Data Classification #Metaheuristic Optimization Algorithms Research #Numerical Analysis (math.NA) #Optimization and Control (math.OC)
paper · pdf · doi:10.48550/arxiv.2011.03151
openalex publication_date 2020/11/05 · openalex created_date 2022/10/01 · openalex updated_date 2026/07/28
Many machine learning solutions are framed as optimization problems which\nrely on good hyperparameters. Algorithms for tuning these hyperparameters\nusually assume access to exact solutions to the underlying learning problem,\nwhich is typically not practical. Here, we apply a recent dynamic accuracy\nderivative-free optimization method to hyperparameter tuning, which allows\ninexact evaluations of the learning problem while retaining convergence\nguarantees. We test the method on the problem of learning elastic net weights\nfor a logistic classifier, and demonstrate its robustness and efficiency\ncompared to a fixed accuracy approach. This demonstrates a promising approach\nfor hyperparameter tuning, with both convergence guarantees and practical\nperformance.\n