2023/01/05 by Zongren Zou, George Em Karniadakis, Zou, Zongren +1 · 2 citations
Computer Science · Materials Science · Physics and Astronomy · #34F05 #62M45 #65L99 #65M99 #65N99 #Computational Physics (physics.comp-ph) #FOS: Computer and information sciences #FOS: Physical sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning in Materials Science #Model Reduction and Neural Networks
paper · pdf · doi:10.48550/arxiv.2301.02152
openalex publication_date 2023/01/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We introduce multi-head neural networks (MH-NNs) to physics-informed machine learning, which is a type of neural networks (NNs) with all nonlinear hidden layers as the body and multiple linear output layers as multi-head. Hence, we construct multi-head physics-informed neural networks (MH-PINNs) as a potent tool for multi-task learning (MTL), generative modeling, and few-shot learning for diverse problems in scientific machine learning (SciML). MH-PINNs connect multiple functions/tasks via a shared body as the basis functions as well as a shared distribution for the head. The former is accomplished by solving multiple tasks with MH-PINNs with each head independently corresponding to each task, while the latter by employing normalizing flows (NFs) for density estimate and generative modeling. To this end, our method is a two-stage method, and both stages can be tackled with standard deep learning tools of NNs, enabling easy implementation in practice. MH-PINNs can be used for various purposes, such as approximating stochastic processes, solving multiple tasks synergistically, providing informative prior knowledge for downstream few-shot learning tasks such as meta-learning and transfer learning, learning representative basis functions, and uncertainty quantification. We demonstrate the effectiveness of MH-PINNs in five benchmarks, investigating also the possibility of synergistic learning in regression analysis. We name the open-source code "Lernaean Hydra" (L-HYDRA), since this mythical creature possessed many heads for performing important multiple tasks, as in the proposed method.