A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems

2019/02/26 by Xuhui Meng, George Em Karniadakis · 703 citations
Decision Sciences · Engineering · Physics and Astronomy · #Algorithm #Artificial intelligence #Artificial neural network #Computer science #Fidelity #Function (biology) #Machine learning #Model Reduction and Neural Networks #Nonlinear system #Nuclear Engineering Thermal-Hydraulics #Physics #Probabilistic and Robust Engineering Design #physics.comp-ph

paper · pdf · doi:10.1016/j.jcp.2019.109020

published in Journal of Computational Physics 401, 109020 (Elsevier BV)

arxiv created 2019/02/26 · openalex publication_date 2019/10/11 · arxiv updated 2020/01/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

We propose a new composite neural network (NN) that can be trained based on multi-fidelity data. It is comprised of three NNs, with the first NN trained using the low-fidelity data and coupled to two high-fidelity NNs, one with activation functions and another one without, in order to discover and exploit nonlinear and linear correlations, respectively, between the low-fidelity and the high-fidelity data. We first demonstrate the accuracy of the new multi-fidelity NN for approximating some standard benchmark functions but also a 20-dimensional function. Subsequently, we extend the recently developed physics-informed neural networks (PINNs) to be trained with multi-fidelity data sets (MPINNs). MPINNs contain four fully-connected neural networks, where the first one approximates the low-fidelity data, while the second and third construct the correlation between the low- and high-fidelity data and produce the multi-fidelity approximation, which is then used in the last NN that encodes the partial differential equations (PDEs). Specifically, in the two high-fidelity NNs a relaxation parameter is introduced, which can be optimized to combine the linear and nonlinear sub-networks. By optimizing this parameter, the present model is capable of learning both the linear and complex nonlinear correlations between the low- and high-fidelity data adaptively. By training the MPINNs, we can:(1) obtain the correlation between the low- and high-fidelity data, (2) infer the quantities of interest based on a few scattered data, and (3) identify the unknown parameters in the PDEs. In particular, we employ the MPINNs to learn the hydraulic conductivity field for unsaturated flows as well as the reactive models for reactive transport. The results demonstrate that MPINNs can achieve relatively high accuracy based on a very small set of high-fidelity data.

Citations

• Predicting the output from a complex computer code when fast approximations are available

Cited by

• Transfer learning based multi-fidelity physics informed deep neural network
• Physics and Equality Constrained Artificial Neural Networks: Application to Forward and Inverse Problems with Multi-fidelity Data Fusion
• Multi-fidelity Bayesian Neural Networks: Algorithms and Applications
• Multifidelity deep operator networks for data-driven and physics-informed problems
• Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data
• Non-invasive inference of thrombus material properties with physics-informed neural networks
• DeepXDE: A Deep Learning Library for Solving Differential Equations
• Harnessing Machine Learning for Hybrid Constitutive Modelling of Viscoelastic Fluid Flows in Computational Rheology
• PPINN: Parareal physics-informed neural network for time-dependent PDEs
• A Kernel-based Resource-efficient Neural Surrogate for Multi-fidelity Prediction of Aerodynamic Field
• MFPC-Net: Multi-fidelity Physics-Constrained Neural Process
• Overcoming the curse of dimensionality for some Hamilton--Jacobi partial differential equations via neural network architectures
• Physics-informed neural networks for improving cerebral hemodynamics predictions
• Multi-Fidelity Delayed Acceptance: hierarchical MCMC sampling for Bayesian inverse problems combining multiple solvers through deep neural networks
• Bridging quantum and classical computing for partial differential equations through multifidelity machine learning
• Assessing the performance of correlation-based multi-fidelity neural emulators
• Transfer learning of neural surrogates on multifidelity groundwater simulations
• Multi-fidelity Gaussian process regression for noisy outputs and non-nested experimental designs: a comparison between the recursive and non-recursive formulations
• Reconstruction of three-dimensional shapes of normal and disease-related erythrocytes from partial observations using multi-fidelity neural networks
• GAN-MDF: A Method for Multi-fidelity Data Fusion in Digital Twins
• The modified Physics-Informed Hybrid Parallel Kolmogorov--Arnold and Multilayer Perceptron Architecture with domain decomposition
• Parameter identification for a damage model using a physics informed neural network
• Personalized Treatment Outcome Prediction from Scarce Data via Dual-Channel Knowledge Distillation and Adaptive Fusion
• Direct Ink Writing: A 3D Printing Technology for Diverse Materials
• A Practitioner's Guide to Kolmogorov-Arnold Networks
• Progressive multi-fidelity learning with neural networks for physical system predictions
• Physics-Informed Machine Learning in Biomedical Science and Engineering
• Multimodal Neural Operators for Real-Time Biomechanical Modelling of Traumatic Brain Injury
• Multi-fidelity neural network–based prediction of tensile strength of high-entropy alloy (FeNiCoCrCu) using molecular dynamics data
• Physics-informed neural networks (PINNs) for fluid mechanics: A review
• Efficient Characterization of Dynamic Response Variation Using Multi-Fidelity Data Fusion through Composite Neural Network
• An adaptive surrogate modeling based on deep neural networks for large-scale Bayesian inverse problems
• Physics-informed neural networks (PINNs) for fluid mechanics: a review
• Bi-fidelity Interpolative Decomposition for Multimodal Data
• Data-Driven Multi-Objective Optimization of Large-Diameter Si Floating-Zone Crystal Growth
• Learning with Mandelbrot and Julia
• D3PINNs: A Novel Physics-Informed Neural Network Framework for Staged Solving of Time-Dependent Partial Differential Equations
• Projection-based multifidelity linear regression for data-scarce applications
• Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems
• On Some Tunable Multi-fidelity Bayesian Optimization Frameworks
• Fusing CFD and measurement data using transfer learning
• Physics Constrained Learning for Data-driven Inverse Modeling from Sparse Observations
• Surrogate Modeling of Landau Damping with Deep Operator Networks
• Single- to multi-fidelity history-dependent learning with uncertainty quantification and disentanglement: application to data-driven constitutive modeling
• A hierarchical neural hybrid method for failure probability estimation
• Improved Surrogate Modeling of Fluid Dynamics with Physics-Informed Neural Networks
• On the performance of multi-fidelity and reduced-dimensional neural emulators for inference of physiological boundary conditions
• Physics-Informed DeepONets for drift-diffusion on metric graphs: simulation and parameter identification
• BridgeNet: A Hybrid, Physics-Informed Machine Learning Framework for Solving High-Dimensional Fokker-Planck Equations
• Physics-informed machine learning
• Uncertainty Quantification of Locally Nonlinear Dynamical Systems using Neural Networks
• Bayesian and non-Bayesian multi-fidelity surrogate models for multi-objective aerodynamic optimization under extreme cost imbalance
• Extremum Seeking for PDE Systems using Physics-Informed Neural Networks
• How machine learning can help the design and analysis of composite materials and structures?
• A New Artificial Neuron Proposal with Trainable Simultaneous Local and Global Activation Function
• A transfer learning metamodel using artificial neural networks applied to natural convection flows in enclosures
• Benchmarking Multi-fidelity Neural Operators on Complex PDE Problems with Non-trivial Fidelity Differences
• On-the-Fly Adaptivity for Nonlinear Twoscale Simulations Using Artificial Neural Networks and Reduced Order Modeling
• Conformalized-KANs: Uncertainty Quantification with Coverage Guarantees for Kolmogorov-Arnold Networks (KANs) in Scientific Machine Learning
• Inverse design optimization framework via a two-step deep learning approach: application to a wind turbine airfoil
• Learning properties of ordered and disordered materials from multi-fidelity data
• Multi-Fidelity Aerodynamic Data Fusion with a Deep Neural Network Modeling Method. [europepmc]
• Learning functionals via LSTM neural networks for predicting vessel dynamics in extreme sea states. [europepmc]
• Integration of Machine Learning and Coarse-Grained Molecular Simulations for Polymer Materials: Physical Understandings and Molecular Design. [europepmc]
• Bridging Fidelities to Predict Nanoindentation Tip Radii Using Interpretable Deep Learning Models. [europepmc]
• Machine Learning Methods for Multiscale Physics and Urban Engineering Problems. [europepmc]
• Influence of land-sea breeze on PM[Formula: see text] prediction in central and southern Taiwan using composite neural network. [europepmc]
• Using satellite data on remote transportation of air pollutants for PM2.5 prediction in northern Taiwan. [europepmc]
• Graph Laplacian-based spectral multi-fidelity modeling. [europepmc]
• h-Analysis and data-parallel physics-informed neural networks. [europepmc]
• Internal and External Pipe Defect Characterization via High-Frequency Lamb Waves Generated by Unidirectional EMAT. [europepmc]
• Connecting Structural Characteristics and Material Properties in Phase-Separating Polymer Solutions: Phase-Field Modeling and Physics-Informed Neural Networks. [europepmc]
• Transfer learning with graph neural networks for improved molecular property prediction in the multi-fidelity setting. [europepmc]
• Enhancing Brain-Computer Interfaces through Kriging-Based Fusion of Sparse Regression Partial Differential Equations to Counter Injection Molding View of Node Displacement Effects. [europepmc]
• Physics-informed two-tier neural network for non-linear model order reduction. [europepmc]
• Physics-informed machine learning for automatic model reduction in chemical reaction networks. [europepmc]
• Effects of Surrogate Hybridization and Adaptive Sampling for Simulation-Based Optimization. [europepmc]
• A Method for Calculating Residual Strength of Crack Arrest Hole on Tungsten-Copper Functionally Graded Materials by Phase-Field Gradient Element Combined with Multi-Fidelity Neural Network. [europepmc]
• A Multi-Fidelity Data Fusion Approach Based on Semi-Supervised Learning for Image Super-Resolution in Data-Scarce Scenarios. [europepmc]
• Multifidelity deep learning modeling of spatiotemporal lung mechanics. [europepmc]
• Multifidelity digital twin for real-time monitoring of structural dynamics in aquaculture net cages. [europepmc]
• Translating Mechanistic Insights Into Action and Revealing New Pathways: Machine Learning Approaches in Heart Failure With Preserved Ejection Fraction. [europepmc]
• Patient-specific hemodynamic simulation for left ventricular assist device via non-invasive monitoring based on lumped parameter model and hierarchical neural network. [europepmc]

Related

• Feature-adjacent multi-fidelity physics-informed machine learning for partial differential equations
• MFPC-Net: Multi-fidelity Physics-Constrained Neural Process
• Transfer Learning on Multi-Fidelity Data
• Data-Guided Physics-Informed Neural Networks for Solving Inverse Problems in Partial Differential Equations
• Efficient PINNs via Multi-Head Unimodular Regularization of the Solutions Space