Deep connections between learning from limited labels & physical parameter estimation -- inspiration for regularization

2020/03/17 by Bas Peters, Peters, Bas
Computer Science · #68T45 #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #FOS: Physical sciences #Geophysics (physics.geo-ph) #I.2.10 #I.4.6 #Image Processing and 3D Reconstruction #Image and Video Processing (eess.IV) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2003.07908

openalex publication_date 2020/03/17 · openalex created_date 2024/04/10 · openalex updated_date 2026/07/28

Abstract

Recently established equivalences between differential equations and the structure of neural networks enabled some interpretation of training of a neural network as partial-differential-equation (PDE) constrained optimization. We add to the previously established connections, explicit regularization that is particularly beneficial in the case of single large-scale examples with partial annotation. We show that explicit regularization of model parameters in PDE constrained optimization translates to regularization of the network output. Examination of the structure of the corresponding Lagrangian and backpropagation algorithm do not reveal additional computational challenges. A hyperspectral imaging example shows that minimum prior information together with cross-validation for optimal regularization parameters boosts the segmentation accuracy.

Related

• Solving Differential Equations with Constrained Learning
• PINNverse: Accurate parameter estimation in differential equations from noisy data with constrained physics-informed neural networks
• Inverse Evolution Layers: Physics-informed Regularizers for Deep Neural Networks
• Partial differential equation regularization for supervised machine\n learning
• Characterizing and Mitigating the Difficulty in Training Physics-informed Artificial Neural Networks under Pointwise Constraints