Improving physics-informed neural networks with meta-learned optimization

2023/03/13 by Bihlo, Alex · 3 citations
#FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Numerical Analysis (math.NA)

paper · doi:10.48550/arxiv.2303.07127

Abstract

We show that the error achievable using physics-informed neural networks for solving systems of differential equations can be substantially reduced when these networks are trained using meta-learned optimization methods rather than to using fixed, hand-crafted optimizers as traditionally done. We choose a learnable optimization method based on a shallow multi-layer perceptron that is meta-trained for specific classes of differential equations. We illustrate meta-trained optimizers for several equations of practical relevance in mathematical physics, including the linear advection equation, Poisson's equation, the Korteweg--de Vries equation and Burgers' equation. We also illustrate that meta-learned optimizers exhibit transfer learning abilities, in that a meta-trained optimizer on one differential equation can also be successfully deployed on another differential equation.

Cited by

• Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equation
• Low-rank adaptive physics-informed HyperDeepONets for solving differential equations
• Latent Representation Learning of Multi-scale Thermophysics: Application to Dynamics in Shocked Porous Energetic Material

Related

• Metamizer: a versatile neural optimizer for fast and accurate physics simulations
• Adaptive Physics-informed Neural Networks: A Survey
• Meta-learning of Physics-informed Neural Networks for Efficiently Solving Newly Given PDEs
• Solving differential equations using physics informed deep learning: a hand-on tutorial with benchmark tests
• Hypernetwork-based Meta-Learning for Low-Rank Physics-Informed Neural Networks