New Heuristics for Parallel and Scalable Bayesian Optimization

2018/07/01 by Ran Rubin, Rubin, Ran
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #Advanced Multi-Objective Optimization Algorithms #Applications (stat.AP) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms

paper · pdf · doi:10.48550/arxiv.1807.00373

openalex publication_date 2018/07/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Bayesian optimization has emerged as a strong candidate tool for global optimization of functions with expensive evaluation costs. However, due to the dynamic nature of research in Bayesian approaches, and the evolution of computing technology, using Bayesian optimization in a parallel computing environment remains a challenge for the non-expert. In this report, I review the state-of-the-art in parallel and scalable Bayesian optimization methods. In addition, I propose practical ways to avoid a few of the pitfalls of Bayesian optimization, such as oversampling of edge parameters and over-exploitation of high performance parameters. Finally, I provide relatively simple, heuristic algorithms, along with their open source software implementations, that can be immediately and easily deployed in any computing environment.

Citations

• Practical Bayesian Optimization of Machine Learning Algorithms
• Thompson Sampling for Contextual Bandits with Linear Payoffs
• Taking the Human Out of the Loop: A Review of Bayesian Optimization
• Scalable iterative methods for sampling from massive Gaussian random vectors
• Parallel and Distributed Thompson Sampling for Large-scale Accelerated\n Exploration of Chemical Space
• Variational Inference: A Review for Statisticians

Related

• Simple and Scalable Parallelized Bayesian Optimization
• Parallel Bayesian Global Optimization of Expensive Functions
• Scalable Constrained Bayesian Optimization
• Parallel Mixed Bayesian Optimization Algorithm: A Scaleup Analysis
• Parallel Bayesian Global Optimization of Expensive Functions