Understanding and Mitigating Gradient Flow Pathologies in Physics-Informed Neural Networks

2021/01/01 by Sifan Wang, Yujun Teng, Paris Perdikaris · 55 citations
Engineering · Physics and Astronomy · #Fluid Dynamics and Turbulent Flows #Fluid Dynamics and Vibration Analysis #Model Reduction and Neural Networks

paper · doi:10.1137/20m1318043

openalex publication_date 2021/01/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/31

Abstract

Related DatabasesWeb of Science You must be logged in with an active subscription to view this.Article DataHistorySubmitted: 12 February 2020Accepted: 25 May 2021Published online: 09 September 2021Keywordsdeep learning, differential equations, optimization, stiff dynamics, computational physicsAMS Subject Headings68T99, 65M99, 68U20Publication DataISSN (print): 1064-8275ISSN (online): 1095-7197Publisher: Society for Industrial and Applied MathematicsCODEN: sjoce3

Citations

• On the Construction and Comparison of Difference Schemes
• Artificial neural networks for solving ordinary and partial differential equations
• Understanding deep convolutional networks
• Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
• Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data
• Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations

Cited by

• Lang-PINN: From Language to Physics-Informed Neural Networks via a Multi-Agent Framework
• Enhancing Training of Physics-Informed Neural Networks Using Domain Decomposition–Based Preconditioning Strategies
• Physics-constrained neural networks for half-space seismic wave modeling
• Stiff neural ordinary differential equations
• The Hard-Constraint PINNs for Interface Optimal Control Problems
• Mixture-of-Experts Operator Transformer for Large-Scale PDE Pre-Training
• Learning Low Rank Neural Representations of Hyperbolic Wave Dynamics from Data
• A Practitioner's Guide to Kolmogorov-Arnold Networks
• Training Deep Physics-Informed Kolmogorov-Arnold Networks
• Learning Optimal Power Flow with Pointwise Constraints
• Iterative Training of Physics-Informed Neural Networks with Fourier-enhanced Features
• MNO: Multiscale Neural Operator for Computational Fluid Dynamics with 3D Point Cloud Data
• Causality-guided adaptive sampling method for physics-informed neural networks solving forward problems of partial differential equations
• Operator-Consistent Physics-Informed Learning for Wafer Thermal Reconstruction in Lithography
• AB-PINNs: Adaptive-Basis Physics-Informed Neural Networks for Residual-Driven Domain Decomposition
• AutoBalance: An Automatic Balancing Framework for Training Physics-Informed Neural Networks
• Physics-Informed Machine Learning in Biomedical Science and Engineering
• Differentiable physics for sound field reconstruction
• Nyström-Accelerated Primal LS-SVMs: Breaking the O(an³) Complexity Bottleneck for Scalable ODEs Learning
• Robust and efficient solvers for nonlinear partial differential equations based on random feature method
• Physics-Informed Neural Operator for Learning Partial Differential Equations
• On the joint observability of flow fields and particle properties from Lagrangian trajectories: evidence from neural data assimilation
• Enhancing PINN Performance Through Lie Symmetry Group
• Learning Hamiltonian Dynamics at Scale: A Differential-Geometric Approach
• Reparameterizing 4DVAR with neural fields
• MATLAB Implementation of Physics Informed Deep Neural Networks for Forward and Inverse Structural Vibration Problems
• Learning the solution operator of parametric partial differential equations with physics-informed DeepONets
• Choose a Transformer: Fourier or Galerkin
• Energy minimisation using overlapping tensor-product free-knot B-splines
• LVADNet3D: A Deep Autoencoder for Reconstructing 3D Intraventricular Flow from Sparse Hemodynamic Data
• A Variational Framework for Residual-Based Adaptivity in Neural PDE Solvers and Operator Learning
• Semi-Discrete in Time Method for Time-Dependent Equations by Random Neural Basis
• Multi-Objective Loss Balancing in Physics-Informed Neural Networks for Fluid Flow Applications
• Discovery of Unstable Singularities
• PINGS: Physics-Informed Neural Network for Fast Generative Sampling
• Physics-informed sensor coverage through structure preserving machine learning
• Physics-Informed Neural Networks vs. Physics Models for Non-Invasive Glucose Monitoring: A Comparative Study Under Noise-Stressed Synthetic Conditions
• ReBaNO: Reduced Basis Neural Operator Mitigating Generalization Gaps and Achieving Discretization Invariance
• Continuous-Time Value Iteration for Multi-Agent Reinforcement Learning
• Physics-Informed Neural Networks in Clean Combustion: A Pathway to Sustainable Aerospace Propulsion
• HyPINO: Multi-Physics Neural Operators via HyperPINNs and the Method of Manufactured Solutions
• Learning functions through Diffusion Maps
• Fast PDE-constrained optimization via self-supervised operator learning
• D3PINNs: A Novel Physics-Informed Neural Network Framework for Staged Solving of Time-Dependent Partial Differential Equations
• Conditionally adaptive augmented Lagrangian method for physics-informed learning of forward and inverse problems
• Eig-PIELM: A Mesh-Free Approach for Efficient Eigen-Analysis with Physics-Informed Extreme Learning Machines
• Data-driven particle dynamics: Structure-preserving coarse-graining for emergent behavior in non-equilibrium systems
• Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next
• SSBE-PINN: A Sobolev Boundary Scheme Boosting Stability and Accuracy in Elliptic/Parabolic PDE Learning
• Structure-Preserving Digital Twins via Conditional Neural Whitney Forms
• Fast, Convex and Conditioned Network for Multi-Fidelity Vectors and Stiff Univariate Differential Equations
• Improved Training Strategies for Physics-Informed Neural Networks using Real Experimental Data in Aluminum Spot Welding
• Solved in Unit Domain: JacobiNet for Differentiable Coordinate-Transformed PINNs
• Separated-Variable Spectral Neural Networks: A Physics-Informed Learning Approach for High-Frequency PDEs
• Physics-Informed PointNets for Modeling Electromagnetic Scattering from All-Dielectric Metasurfaces with Inclined Nanopillars

Related

• Improved physics-informed neural network in mitigating gradient related failures
• Understanding and mitigating gradient pathologies in physics-informed neural networks
• GradINN: Gradient Informed Neural Network
• Energy stable neural network for gradient flow equations
• Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspective