Bayesian Optimization under Uncertainty for Training a Scale Parameter in Stochastic Models

2025/10/07 by Yadav, Akash, Zhang, Ruda · 1 citation
#Computational Engineering #FOS: Computer and information sciences #FOS: Mathematics #Finance #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #and Science (cs.CE)

paper · doi:10.48550/arxiv.2510.06439

Abstract

Hyperparameter tuning is a challenging problem especially when the system itself involves uncertainty. Due to noisy function evaluations, optimization under uncertainty can be computationally expensive. In this paper, we present a novel Bayesian optimization framework tailored for hyperparameter tuning under uncertainty, with a focus on optimizing a scale- or precision-type parameter in stochastic models. The proposed method employs a statistical surrogate for the underlying random variable, enabling analytical evaluation of the expectation operator. Moreover, we derive a closed-form expression for the optimizer of the random acquisition function, which significantly reduces computational cost per iteration. Compared with a conventional one-dimensional Monte Carlo-based optimization scheme, the proposed approach requires 40 times fewer data points, resulting in up to a 40-fold reduction in computational cost. We demonstrate the effectiveness of the proposed method through two numerical examples in computational engineering.

Citations

• Sampling from Gaussian Processes: A Tutorial and Applications in Global Sensitivity Analysis and Optimization
• Stochastic Subspace via Probabilistic Principal Component Analysis for Characterizing Model Error
• Optimizing Posterior Samples for Bayesian Optimization via Rootfinding
• Automated design of nonreciprocal thermal emitters via Bayesian optimization
• Multi-fidelity Bayesian Optimization: A Review
• Efficient Robust Bayesian Optimization for Arbitrary Uncertain Inputs
• Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020
• Bayesian optimisation under uncertain inputs
• Learning to Optimize Via Posterior Sampling
• Learning to Optimize via Posterior Sampling
• Practical Bayesian Optimization of Machine Learning Algorithms
• Theory and Applications of Robust Optimization
• Robust optimization ? methodology and applications

Cited by

• Nonparametric Stochastic Subspaces via the Bootstrap for Characterizing Model Error

Related

• Towards Automatic Bayesian Optimization: A first step involving acquisition functions
• A Hybrid Monte Carlo Architecture for Parameter Optimization
• Pseudo-Bayesian Optimization
• Enhancing Gaussian Process Surrogates for Optimization and Posterior Approximation via Random Exploration
• Informed Initialization for Bayesian Optimization and Active Learning