A survey on multi-objective hyperparameter optimization algorithms for\n Machine Learning

2021/11/23 by Alejandro Morales-Hernández, Morales-Hernández, Alejandro, Inneke Van Nieuwenhuyse +3 · 4 citations
Computer Science · Engineering · #Advanced Multi-Objective Optimization Algorithms #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #FOS: Mathematics #Heat Transfer and Optimization #Machine Learning (cs.LG) #Metaheuristic Optimization Algorithms Research #Optimization and Control (math.OC)

paper · pdf · doi:10.48550/arxiv.2111.13755

openalex publication_date 2021/11/23 · openalex created_date 2022/11/14 · openalex updated_date 2026/07/28

Abstract

Hyperparameter optimization (HPO) is a necessary step to ensure the best\npossible performance of Machine Learning (ML) algorithms. Several methods have\nbeen developed to perform HPO; most of these are focused on optimizing one\nperformance measure (usually an error-based measure), and the literature on\nsuch single-objective HPO problems is vast. Recently, though, algorithms have\nappeared that focus on optimizing multiple conflicting objectives\nsimultaneously. This article presents a systematic survey of the literature\npublished between 2014 and 2020 on multi-objective HPO algorithms,\ndistinguishing between metaheuristic-based algorithms, metamodel-based\nalgorithms, and approaches using a mixture of both. We also discuss the quality\nmetrics used to compare multi-objective HPO procedures and present future\nresearch directions.\n

Cited by

• Machine learning–aided design of La-based composite modified biochar: Efficient materials and cost optimization for low-phosphorus water treatment
• Reward driven discovery of the optimal microstructure representations with invariant variational autoencoders
• Development of a Multiprocessing Interface Genetic Algorithm for Optimising a Multilayer Perceptron for Disease Prediction
• Interpreting Graph Inference with Skyline Explanations

Related

• Multi-objective hyperparameter optimization with performance uncertainty
• On hyperparameter optimization of machine learning algorithms: Theory and practice
• Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges
• Hyperparameter Optimization in Machine Learning
• Parallel Multi-Objective Hyperparameter Optimization with Uniform Normalization and Bounded Objectives