High Dimensional Level Set Estimation with Bayesian Neural Network

2020/12/17 by Huong Ha, Ha, Huong, Sunil Gupta +5 · 2 citations
Computer Science · Engineering · #Advanced Multi-Objective Optimization Algorithms #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Reservoir Engineering and Simulation Methods

paper · pdf · doi:10.48550/arxiv.2012.09973

openalex publication_date 2020/12/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Level Set Estimation (LSE) is an important problem with applications in various fields such as material design, biotechnology, machine operational testing, etc. Existing techniques suffer from the scalability issue, that is, these methods do not work well with high dimensional inputs. This paper proposes novel methods to solve the high dimensional LSE problems using Bayesian Neural Networks. In particular, we consider two types of LSE problems: (1) explicit LSE problem where the threshold level is a fixed user-specified value, and, (2) implicit LSE problem where the threshold level is defined as a percentage of the (unknown) maximum of the objective function. For each problem, we derive the corresponding theoretic information based acquisition function to sample the data points so as to maximally increase the level set accuracy. Furthermore, we also analyse the theoretical time complexity of our proposed acquisition functions, and suggest a practical methodology to efficiently tune the network hyper-parameters to achieve high model accuracy. Numerical experiments on both synthetic and real-world datasets show that our proposed method can achieve better results compared to existing state-of-the-art approaches.

Citations

• Bayesian Active Learning for Classification and Preference Learning
• A Practical Bayesian Framework for Backpropagation Networks
• Practical Bayesian Optimization of Machine Learning Algorithms
• Taking the Human Out of the Loop: A Review of Bayesian Optimization
• Truncated Variance Reduction: A Unified Approach to Bayesian Optimization and Level-Set Estimation
• Stagewise Safe Bayesian Optimization with Gaussian Processes
• Robust Super-Level Set Estimation using Gaussian Processes
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Bayesian Experimental Design for Finding Reliable Level Set under Input Uncertainty

Cited by

• Multi-Metric Adaptive Experimental Design Under a Fixed Budget with Validation
• Vendi Information Gain: An Alternative To Mutual Information For Science And Machine Learning

Related

• Amortized Bayesian Multilevel Models
• Bayesian neural networks and dimensionality reduction
• Bayesian Deep Learning with Multilevel Trace-class Neural Networks
• Scalable Bayesian neural networks by layer-wise input augmentation
• Bayesian Learning of Neural Network Architectures