Tackling the curse of dimensionality with physics-informed neural networks

2023/07/23 by Zheyuan Hu, Khemraj Shukla, George Em Karniadakis +1 · 1 voice · 160 citations
Engineering · Physics and Astronomy · #Artificial intelligence #Artificial neural network #Computer science #Curse #Curse of dimensionality #Machine learning #Model Reduction and Neural Networks #Nuclear Engineering Thermal-Hydraulics #Nuclear reactor physics and engineering #Physics #Sociology #Statistical physics

paper · pdf · open access · doi:10.1016/j.neunet.2024.106369

published in Neural Networks 176, 106369 (Elsevier BV)

openalex publication_date 2024/05/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/03

Abstract

The curse-of-dimensionality taxes computational resources heavily with exponentially increasing computational cost as the dimension increases. This poses great challenges in solving high-dimensional partial differential equations (PDEs), as Richard E. Bellman first pointed out over 60 years ago. While there has been some recent success in solving numerical PDEs in high dimensions, such computations are prohibitively expensive, and true scaling of general nonlinear PDEs to high dimensions has never been achieved. We develop a new method of scaling up physics-informed neural networks (PINNs) to solve arbitrary high-dimensional PDEs. The new method, called Stochastic Dimension Gradient Descent (SDGD), decomposes a gradient of PDEs' and PINNs' residual into pieces corresponding to different dimensions and randomly samples a subset of these dimensional pieces in each iteration of training PINNs. We prove theoretically the convergence and other desired properties of the proposed method. We demonstrate in various diverse tests that the proposed method can solve many notoriously hard high-dimensional PDEs, including the Hamilton-Jacobi-Bellman (HJB) and the Schrödinger equations in tens of thousands of dimensions very fast on a single GPU using the PINNs mesh-free approach. Notably, we solve nonlinear PDEs with nontrivial, anisotropic, and inseparable solutions in less than one hour for 1000 dimensions and in 12 h for 100,000 dimensions on a single GPU using SDGD with PINNs. Since SDGD is a general training methodology of PINNs, it can be applied to any current and future variants of PINNs to scale them up for arbitrary high-dimensional PDEs.

Cited by

• Stochastic Dimension Zeroth-Order Estimator: Stable and Memory-Efficient Training of PINNs
• Neural Very Weak Formulations enabling Hardware-Oriented deep PDE solvers
• Physics-Informed Neural Networks for Joint Source and Parameter Estimation in Advection-Diffusion Equations
• Evolutionary Computation for Sparse Multi-Objective Optimization: A Survey
• A Physics Informed Machine Learning Framework for Optimal Sensor Placement and Parameter Estimation
• Semantic-driven Wireless Environment Knowledge Representation for Efficiency-Accuracy Balanced Beam Prediction in Vehicular Networks
• HEATNETs: Explainable Random Feature Neural Networks for High-Dimensional Parabolic PDEs
• NeuroPDE+: A Scalable Neuromorphic PDE Accelerator Based on Spintronic and Ferroelectric Devices
• Efficient Global-Local Fusion Sampling for Physics-Informed Neural Networks
• A Practitioner's Guide to Kolmogorov-Arnold Networks
• A Variational Framework for the Algorithmic Complexity of PDE Solutions
• Iterative Training of Physics-Informed Neural Networks with Fourier-enhanced Features
• A derivative-free localized stochastic method for very high-dimensional semilinear parabolic PDEs
• Examining the robustness of Physics-Informed Neural Networks to noise for Inverse Problems
• Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker–Planck Equations
• A Variational Framework for Residual-Based Adaptivity in Neural PDE Solvers and Operator Learning
• Error analysis for the deep Kolmogorov method
• Physics-Informed Neural Networks in Clean Combustion: A Pathway to Sustainable Aerospace Propulsion
• Theory Foundation of Physics-Enhanced Residual Learning
• Sum-of-Gaussians tensor neural networks for high-dimensional Schrödinger equation
• BubbleOKAN: A Physics-Informed Interpretable Neural Operator for High-Frequency Bubble Dynamics
• Low-Rank Augmented Implicit Neural Representation for Unsupervised High-Dimensional Quantitative MRI Reconstruction
• Quasi-Random Physics-informed Neural Networks
• NeuroPDE: A Neuromorphic PDE Solver Based on Spintronic and Ferroelectric Devices
• Neural Tangent Kernel Analysis to Probe Convergence in Physics-informed Neural Solvers: PIKANs vs. PINNs
• Deep random difference method for high-dimensional quasilinear parabolic partial differential equations
• Improving Memory Efficiency for Training KANs via Meta Learning
• A deep shotgun method for solving high-dimensional parabolic partial differential equations
• Physics-informed neural network framework for solving forward and inverse flexoelectric problems
• IIKL: Isometric Immersion Kernel Learning with Riemannian Manifold for Geometric Preservation
• Anant-Net: Breaking the Curse of Dimensionality with Scalable and Interpretable Neural Surrogate for High-Dimensional PDEs
• Dual Natural Gradient Descent for Scalable Training of Physics-Informed Neural Networks
• Finding separatrices of dynamical flows with Deep Koopman Eigenfunctions
• Accelerating Multivariate Newton Interpolation in Downward Closed Polynomial Spaces
• Beyond Accuracy: EcoL2 Metric for Sustainable Neural PDE Solvers
• Stochastic-Dimension Frozen Sampled Neural Network for High-Dimensional Gross-Pitaevskii Equations on Unbounded Domains
• Deep Neural networks for solving high-dimensional parabolic partial differential equations
• Universal Approximation of Nonlinear Operators and Their Derivatives
• Neural semi-Lagrangian method for high-dimensional advection-diffusion problems
• Energy-Based Deep Learning Initialization Method Achieves Precise Evolution of Soliton Dynamics
• Enhancing sea surface salinity short-term prediction using physically informed deep learning
• Neural Network Element Method for Partial Differential Equations

Discussions

• Tackling the curse of dimensionality with physics-informed neural networks [hn, 77 points, 17 comments]

Related

• Approximation Theory and Applications of Randomized Neural Networks for Solving High-Dimensional PDEs
• An Extreme Learning Machine-Based Method for Computational PDEs in Higher Dimensions
• Deep Neural networks for solving high-dimensional parabolic partial differential equations
• Forward-Backward Stochastic Neural Networks: Deep Learning of High-dimensional Partial Differential Equations
• Anant-Net: Breaking the Curse of Dimensionality with Scalable and Interpretable Neural Surrogate for High-Dimensional PDEs