Blending Neural Operators and Relaxation Methods in PDE Numerical Solvers

2022/08/28 by Enrui Zhang, Zhang, Enrui, Adar Kahana +10 · 20 citations
Computer Science · Mathematics · Physics and Astronomy · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Matrix Theory and Algorithms #Model Reduction and Neural Networks #Numerical Analysis (math.NA) #Numerical methods for differential equations

paper · pdf · doi:10.48550/arxiv.2208.13273

openalex publication_date 2022/08/28 · openalex created_date 2022/08/31 · openalex updated_date 2026/07/28

Abstract

Neural networks suffer from spectral bias having difficulty in representing the high frequency components of a function while relaxation methods can resolve high frequencies efficiently but stall at moderate to low frequencies. We exploit the weaknesses of the two approaches by combining them synergistically to develop a fast numerical solver of partial differential equations (PDEs) at scale. Specifically, we propose HINTS, a hybrid, iterative, numerical, and transferable solver by integrating a Deep Operator Network (DeepONet) with standard relaxation methods, leading to parallel efficiency and algorithmic scalability for a wide class of PDEs, not tractable with existing monolithic solvers. HINTS balances the convergence behavior across the spectrum of eigenmodes by utilizing the spectral bias of DeepONet, resulting in a uniform convergence rate and hence exceptional performance of the hybrid solver overall. Moreover, HINTS applies to large-scale, multidimensional systems, it is flexible with regards to discretizations, computational domain, and boundary conditions.

Cited by

• ANCHOR: Error-Controlled Adaptive Numerical Correction for Neural Operator Time Marching
• Nodal Hybrid Neural Solvers for Parametric PDE Systems
• A Neural-Operator Preconditioned Newton Method for Accelerated Nonlinear Solvers
• Neural-Initialized Newton: Accelerating Nonlinear Finite Elements via Operator Learning
• NOWS: Neural Operator Warm Starts for Accelerating Iterative Solvers
• Learning Sparse Approximate Inverse Preconditioners for Conjugate Gradient Solvers on GPUs
• A Greedy PDE Router for Blending Neural Operators and Classical Methods
• Learning Singularity-Encoded Green's Functions with Application to Iterative Methods
• Accurate and scalable deep Maxwell solvers using multilevel iterative methods
• Estimating Parameter Fields in Multi-Physics PDEs from Scarce Measurements
• Leveraging Operator Learning to Accelerate Convergence of the Preconditioned Conjugate Gradient Method
• Graph Lineages and Skeletal Graph Products
• Accelerating Conjugate Gradient Solvers for Homogenization Problems with Unitary Neural Operators
• Fredholm Neural Networks for inverse problems in elliptic PDEs
• RAPNet: Accelerating Algebraic Multigrid with Learned Sparse Corrections
• PCG-Informed Neural Solvers for High-Resolution Homogenization of Periodic Microstructures
• Hybrid deep learning and iterative methods for accelerated solutions of viscous incompressible flow
• Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations
• Reformulating Neural Operators in d+1 Dimensions for Embedding Evolution
• PaRO-DeepONet: a particle-informed reduced-order deep operator network for Poisson solver in PIC simulations

Related

• Learning Neural PDE Solvers with Convergence Guarantees
• A novel paradigm for solving PDEs: multi scale neural computing
• DRIFT-Net: A Spectral--Coupled Neural Operator for PDEs Learning
• PDE Solvers Should Be Local: Fast, Stable Rollouts with Learned Local Stencils
• A Greedy PDE Router for Blending Neural Operators and Classical Methods