2021/02/09 by M. Torabi Rad, Mahdi Rad, Alexandre Viardin +6
Computer Science · Engineering · Mathematics · Physics and Astronomy · #Algorithm #Applied Physics (physics.app-ph) #Applied mathematics #Artificial intelligence #Artificial neural network #Computer science #Differential equation #Discretization #FOS: Computer and information sciences #FOS: Physical sciences #Heat Transfer and Optimization #Inference #Limit (mathematics) #Machine Learning (cs.LG) #Mathematical analysis #Mathematical optimization #Mathematics #Model Reduction and Neural Networks #Nanofluid Flow and Heat Transfer #Regularization (linguistics) #Theoretical computer science #Training (meteorology) #cs.LG #physics.app-ph
paper · pdf · doi:10.48550/arxiv.2102.04890
openalex publication_date 2021/02/09 · arxiv created 2021/02/10 · arxiv updated 2021/02/11 · openalex created_date 2021/02/15 · openalex updated_date 2026/07/28
Deep neural networks are transforming fields ranging from computer vision to computational medicine, and we recently extended their application to the field of phase-change heat transfer by introducing theory-trained neural networks (TTNs) for a solidification problem \citeTTN. Here, we present general, in-depth, and empirical insights into theory-training networks for learning the solution of highly coupled differential equations. We analyze the deteriorating effects of the oscillating loss on the ability of a network to satisfy the equations at the training data points, measured by the final training loss, and on the accuracy of the inferred solution. We introduce a theory-training technique that, by leveraging regularization, eliminates those oscillations, decreases the final training loss, and improves the accuracy of the inferred solution, with no additional computational cost. Then, we present guidelines that allow a systematic search for the network that has the optimal training time and inference accuracy for a given set of equations; following these guidelines can reduce the number of tedious training iterations in that search. Finally, a comparison between theory-training and the rival, conventional method of solving differential equations using discretization attests to the advantages of theory-training not being necessarily limited to high-dimensional sets of equations. The comparison also reveals a limitation of the current theory-training framework that may limit its application in domains where extreme accuracies are necessary.