Noisy PDE Training Requires Bigger PINNs

2025/07/09 by Sebastien Andre-Sloan, Andre-Sloan, Sebastien, Anirbit Mukherjee +3
Computer Science · Physics and Astronomy · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2507.06967

openalex publication_date 2025/07/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Physics-Informed Neural Networks (PINNs) are increasingly used to approximate solutions of partial differential equations (PDEs), particularly in high dimensions. In real-world settings, data are often noisy, making it crucial to understand when a predictor can still achieve low empirical risk. Yet, little is known about the conditions under which a PINN can do so effectively. We analyse PINNs applied to the Hamilton--Jacobi--Bellman (HJB) PDE and establish a lower bound on the network size required for the supervised PINN empirical risk to fall below the variance of noisy supervision labels. Specifically, if a predictor achieves empirical risk O(η) below σ² (the variance of the supervision data), then necessarily d_Nlog d_N\gtrsim N_s η², where N_s is the number of samples and d_N the number of trainable parameters. A similar constraint holds in the fully unsupervised PINN setting when boundary labels are noisy. Thus, simply increasing the number of noisy supervision labels does not offer a ``free lunch'' in reducing empirical risk. We also give empirical studies on the HJB PDE, the Poisson PDE and the the Navier-Stokes PDE set to produce the Taylor-Green solutions. In these experiments we demonstrate that PINNs indeed need to be beyond a threshold model size for them to train to errors below σ². These results provide a quantitative foundation for understanding parameter requirements when training PINNs in the presence of noisy data.

Citations

• Convergence and error control of consistent PINNs for elliptic PDEs
• Impact of Computation in Integral Reinforcement Learning for Continuous-Time Control
• Bias-Variance Trade-off in Physics-Informed Neural Networks with Randomized Smoothing for High-Dimensional PDEs
• Uncertainty quantification for noisy inputs-outputs in physics-informed neural networks and neural operators
• Is L² Physics-Informed Loss Always Suitable for Training Physics-Informed Neural Network?
• Generic bounds on the approximation error for physics-informed (and) operator learning
• Learning Physics-Informed Neural Networks without Stacked Back-propagation
• Neural Stochastic PDEs: Resolution-Invariant Learning of Continuous Spatiotemporal Dynamics
• A Universal Law of Robustness via Isoperimetry
• Understanding and Mitigating Gradient Flow Pathologies in Physics-Informed Neural Networks
• When and why PINNs fail to train: A neural tangent kernel perspective
• When and why PINNs fail to train: A neural tangent kernel perspective
• Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs
• Numerical Solution of the Parametric Diffusion Equation by Deep Neural\n Networks
• B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data
• Towards Physics-informed Deep Learning for Turbulent Flow Prediction
• Physics-informed Autoencoders for Lyapunov-stable Fluid Flow Prediction
• A Theoretical Analysis of Deep Neural Networks and Parametric PDEs
• Error bounds for approximations with deep ReLU neural networks in\n W^s,p norms
• Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
• Data-driven discovery of Koopman eigenfunctions for control
• Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach
• Artificial neural networks for solving ordinary and partial differential equations

Related

• Convergence and error analysis of PINNs
• Stability in Training PINNs for Stiff PDEs: Why Initial Conditions Matter
• On the Generalization of PINNs outside the training domain and the Hyperparameters influencing it
• Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs
• Efficient Error Certification for Physics-Informed Neural Networks