Predictive Entropy Search for Efficient Global Optimization of Black-box\n Functions

2014/06/10 by José Miguel Hernández-Lobato, Matthew W. Hoffman, Hernández-Lobato, José Miguel +3 · 29 citations
Computer Science · Decision Sciences · Mathematics · #Advanced Bandit Algorithms Research #Advanced Multi-Objective Optimization Algorithms #Advanced Optimization Algorithms Research #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms

paper · pdf · doi:10.48550/arxiv.1406.2541

openalex publication_date 2014/06/10 · openalex created_date 2025/10/24 · openalex updated_date 2026/07/28

Abstract

We propose a novel information-theoretic approach for Bayesian optimization\ncalled Predictive Entropy Search (PES). At each iteration, PES selects the next\nevaluation point that maximizes the expected information gained with respect to\nthe global maximum. PES codifies this intractable acquisition function in terms\nof the expected reduction in the differential entropy of the predictive\ndistribution. This reformulation allows PES to obtain approximations that are\nboth more accurate and efficient than other alternatives such as Entropy Search\n(ES). Furthermore, PES can easily perform a fully Bayesian treatment of the\nmodel hyperparameters while ES cannot. We evaluate PES in both synthetic and\nreal-world applications, including optimization problems in machine learning,\nfinance, biotechnology, and robotics. We show that the increased accuracy of\nPES leads to significant gains in optimization performance.\n

Citations

• Information-Based Objective Functions for Active Data Selection
• Practical Bayesian Optimization of Machine Learning Algorithms

Cited by

• The Agent Capability Problem: Predicting Solvability Through Information-Theoretic Bounds
• Automated Discovery of Laser Dicing Processes with Bayesian Optimization for Semiconductor Manufacturing
• Local Entropy Search over Descent Sequences for Bayesian Optimization
• Parametric Pareto Set Learning for Expensive Multi-Objective Optimization
• ShaplEIG: Bayesian Experimental Design for Shapley Value Estimation
• Regime-Adaptive Bayesian Optimization via Dirichlet Process Mixtures of Gaussian Processes
• Informed Initialization for Bayesian Optimization and Active Learning
• Beyond Pairwise: Empowering LLM Alignment With Ranked Choice Modeling
• A Guide to Bayesian Optimization in Bioprocess Engineering
• BALLAST: Bayesian Active Learning with Look-ahead Amendment for Sea-drifter Trajectories under Spatio-Temporal Vector Fields
• Beyond Noisy-TVs: Noise-Robust Exploration Via Learning Progress Monitoring
• Information-Theoretic Bayesian Optimization for Bilevel Optimization Problems
• Quantization Algorithms for Random Fourier Features
• A Minimalist Bayesian Framework for Stochastic Optimization
• Scalable Constrained Bayesian Optimization
• Batched Large-scale Bayesian Optimization in High-dimensional Spaces
• Enhancing Trust-Region Bayesian Optimization via Newton Methods
• Gaussian Process Bandit Optimization of the Thermodynamic Variational\n Objective
• Recursive Two-Step Lookahead Expected Payoff for Time-Dependent Bayesian\n Optimization
• On Some Tunable Multi-fidelity Bayesian Optimization Frameworks
• On Simulation and Trajectory Prediction with Gaussian Process Dynamics
• A General Framework for Constrained Bayesian Optimization using Information-based Search
• Parallel Predictive Entropy Search for Multi-objective Bayesian\n Optimization with Constraints
• Uncertainty Quantification for Bayesian Optimization
• Continuous multi-task Bayesian Optimisation with correlation
• BoA-PTA, A Bayesian Optimization Accelerated Error-Free SPICE Solver
• Trusted-Maximizers Entropy Search for Efficient Bayesian Optimization
• Covariance Function Pre-Training with m-Kernels for Accelerated Bayesian Optimisation
• Advancing Bayesian Optimization: The Mixed-Global-Local (MGL) Kernel and Length-Scale Cool Down

Related

• Max-value Entropy Search for Efficient Bayesian Optimization
• Joint Entropy Search for Maximally-Informed Bayesian Optimization
• Parallel Predictive Entropy Search for Batch Global Optimization of\n Expensive Objective Functions
• A Unified Framework for Entropy Search and Expected Improvement in Bayesian Optimization
• Predictive Entropy Search for Bayesian Optimization with Unknown Constraints