The Parallel Knowledge Gradient Method for Batch Bayesian Optimization

2016/06/14 by Peter I. Frazier, Wu, Jian, Frazier, Peter I. · 6 citations
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Machine Learning and Data Classification #Metaheuristic Optimization Algorithms Research

paper · pdf · doi:10.48550/arxiv.1606.04414

openalex publication_date 2016/06/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In many applications of black-box optimization, one can evaluate multiple points simultaneously, e.g. when evaluating the performances of several different neural network architectures in a parallel computing environment. In this paper, we develop a novel batch Bayesian optimization algorithm --- the parallel knowledge gradient method. By construction, this method provides the one-step Bayes-optimal batch of points to sample. We provide an efficient strategy for computing this Bayes-optimal batch of points, and we demonstrate that the parallel knowledge gradient method finds global optima significantly faster than previous batch Bayesian optimization algorithms on both synthetic test functions and when tuning hyperparameters of practical machine learning algorithms, especially when function evaluations are noisy.

Cited by

• Regime-Adaptive Bayesian Optimization via Dirichlet Process Mixtures of Gaussian Processes
• Low-N Protein Activity Optimization with FolDE
• Black-box optimization in immunology and beyond: A practical guide to algorithms and future directions
• Multiobjective Aerodynamic Design Optimization of the NASA Common Research Model
• Fast Bayesian Optimization of Function Networks with Partial Evaluations
• Value-adaptive clinical trial designs for efficient delivery of publicly funded trials - a discussion of methods, case studies, opportunities and challenges

Related

• Simple and Scalable Parallelized Bayesian Optimization
• Preferential Batch Bayesian Optimization
• Discretization-free Knowledge Gradient Methods for Bayesian Optimization
• Efficient computation of the Knowledge Gradient for Bayesian Optimization
• New Heuristics for Parallel and Scalable Bayesian Optimization