Multi-fidelity Bayesian Neural Networks: Algorithms and Applications

2020/12/19 by Xuhui Meng, Hessam Babaee, George Em Karniadakis · 188 citations
Computer Science · Engineering · Mathematics · Physics and Astronomy · #Algorithm #Artificial intelligence #Artificial neural network #Bayesian probability #Computer science #Fidelity #Gaussian Processes and Bayesian Inference #Hyperparameter #Inverse problem #Machine learning #Mathematics #Model Reduction and Neural Networks #Nonlinear system #Physics #Reservoir Engineering and Simulation Methods #cs.LG #physics.comp-ph

paper · pdf · doi:10.1016/j.jcp.2021.110361

published in Journal of Computational Physics 438, 110361 (Elsevier BV) · 31 pages, 11 figures

arxiv created 2020/12/19 · openalex publication_date 2021/04/16 · arxiv updated 2021/06/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

We propose a new class of Bayesian neural networks (BNNs) that can be trained using noisy data of variable fidelity, and we apply them to learn function approximations as well as to solve inverse problems based on partial differential equations (PDEs). These multi-fidelity BNNs consist of three neural networks: The first is a fully connected neural network, which is trained following the maximum a posteriori probability (MAP) method to fit the low-fidelity data; the second is a Bayesian neural network employed to capture the cross-correlation with uncertainty quantification between the low- and high-fidelity data; and the last one is the physics-informed neural network, which encodes the physical laws described by PDEs. For the training of the last two neural networks, we use the Hamiltonian Monte Carlo method to estimate accurately the posterior distributions for the corresponding hyperparameters. We demonstrate the accuracy of the present method using synthetic data as well as real measurements. Specifically, we first approximate a one- and four-dimensional function, and then infer the reaction rates in one- and two-dimensional diffusion-reaction systems. Moreover, we infer the sea surface temperature (SST) in the Massachusetts and Cape Cod Bays using satellite images and in-situ measurements. Taken together, our results demonstrate that the present method can capture both linear and nonlinear correlation between the low- and high-fideilty data adaptively, identify unknown parameters in PDEs, and quantify uncertainties in predictions, given a few scattered noisy high-fidelity data. Finally, we demonstrate that we can effectively and efficiently reduce the uncertainties and hence enhance the prediction accuracy with an active learning approach, using as examples a specific one-dimensional function approximation and an inverse PDE problem.

Citations

• Predicting the output from a complex computer code when fast approximations are available
• A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems

Cited by

• An information field theory approach to Bayesian state and parameter estimation in dynamical systems
• Physics-informed neural networks for improving cerebral hemodynamics predictions
• Multi-Fidelity Delayed Acceptance: hierarchical MCMC sampling for Bayesian inverse problems combining multiple solvers through deep neural networks
• Bridging quantum and classical computing for partial differential equations through multifidelity machine learning
• Reconstruction of three-dimensional shapes of normal and disease-related erythrocytes from partial observations using multi-fidelity neural networks
• Physics-Informed Extreme Learning Machine (PIELM) for Tunnelling-Induced Soil-Pile Interactions
• GenUQ: Predictive Uncertainty Estimates via Generative Hyper-Networks
• On Some Tunable Multi-fidelity Bayesian Optimization Frameworks
• Exploring the Link Between Bayesian Inference and Embodied Intelligence: Toward Open Physical-World Embodied AI Systems
• Single- to multi-fidelity history-dependent learning with uncertainty quantification and disentanglement: application to data-driven constitutive modeling
• MF-GLaM: A multifidelity stochastic emulator using generalized lambda models
• PINN-Obs: Physics-Informed Neural Network-Based Observer for Nonlinear Dynamical Systems
• Physics-informed neural networks combined with polynomial interpolation to solve nonlinear partial differential equations
• Flow field reconstruction from sparse and noisy data based on finite volume physics-informed neural networks
• Bayesian and non-Bayesian multi-fidelity surrogate models for multi-objective aerodynamic optimization under extreme cost imbalance
• Data-Driven Structural State Estimation via Multi-Fidelity Gaussian Process Models
• Towards deployment-centric multimodal AI beyond vision and language
• Graph Laplacian-based spectral multi-fidelity modeling. [europepmc]
• Reconstructing blood flow in data-poor regimes: a vasculature network kernel for Gaussian process regression. [europepmc]
• Research on Spatiotemporal Continuous Information Perception of Overburden Compression-Tensile Strain Transition Zone during Mining and Integrated Safety Guarantee System. [europepmc]
• Multifidelity deep learning modeling of spatiotemporal lung mechanics. [europepmc]
• Multifidelity digital twin for real-time monitoring of structural dynamics in aquaculture net cages. [europepmc]
• A scalable variational method for estimating the latent infection-rate field of an outbreak. [europepmc]