2023/08/18 by Ritam Majumdar, Vishal Jadhav, Majumdar, Ritam +9 · 2 citations
Computer Science · Physics and Astronomy · #Analysis of PDEs (math.AP) #Artificial Intelligence (cs.AI) #Computational Engineering #Computational Physics and Python Applications #FOS: Computer and information sciences #FOS: Mathematics #Finance #Machine Learning (cs.LG) #Model Reduction and Neural Networks #and Science (cs.CE)
paper · pdf · doi:10.48550/arxiv.2308.09290
openalex publication_date 2023/08/18 · openalex created_date 2023/08/22 · openalex updated_date 2026/07/28
Physics-informed neural networks (PINNs) have been widely used to develop neural surrogates for solutions of Partial Differential Equations. A drawback of PINNs is that they have to be retrained with every change in initial-boundary conditions and PDE coefficients. The Hypernetwork, a model-based meta learning technique, takes in a parameterized task embedding as input and predicts the weights of PINN as output. Predicting weights of a neural network however, is a high-dimensional regression problem, and hypernetworks perform sub-optimally while predicting parameters for large base networks. To circumvent this issue, we use a low ranked adaptation (LoRA) formulation to decompose every layer of the base network into low-ranked tensors and use hypernetworks to predict the low-ranked tensors. Despite the reduced dimensionality of the resulting weight-regression problem, LoRA-based Hypernetworks violate the underlying physics of the given task. We demonstrate that the generalization capabilities of LoRA-based hypernetworks drastically improve when trained with an additional physics-informed loss component (HyperPINN) to satisfy the governing differential equations. We observe that LoRA-based HyperPINN training allows us to learn fast solutions for parameterized PDEs like Burger's equation and Navier Stokes: Kovasznay flow, while having an 8x reduction in prediction parameters on average without compromising on accuracy when compared to all other baselines.