Differential Equations for Continuous-Time Deep Learning

2024/01/08 by Lars Ruthotto, Ruthotto, Lars · 3 citations
Mathematics · Physics and Astronomy · #Applied mathematics #Artificial intelligence #Artificial neural network #Computer science #Deep learning #Differential equation #Foundation (evidence) #Law #Machine learning #Mathematical analysis #Mathematics #Model Reduction and Neural Networks #Ode #Ordinary differential equation #Partial differential equation

paper · pdf · doi:10.48550/arxiv.2401.03965

published in arXiv (Cornell University) (Cornell University)

Abstract

This short, self-contained article seeks to introduce and survey continuous-time deep learning approaches that are based on neural ordinary differential equations (neural ODEs). It primarily targets readers familiar with ordinary and partial differential equations and their analysis who are curious to see their role in machine learning. Using three examples from machine learning and applied mathematics, we will see how neural ODEs can provide new insights into deep learning and a foundation for more efficient algorithms.

Cited by

• The Seismic Wavefield Common Task Framework
• The Vanishing Gradient Problem for Stiff Neural Differential Equations
• Weight-Parameterization in Continuous Time Deep Neural Networks for Surrogate Modeling

Related

• Dissecting Neural ODEs
• Neural Differential Equations for Learning to Program Neural Nets Through Continuous Learning Rules
• Continuous Convolutional Neural Networks: Coupled Neural PDE and ODE
• Neural Ordinary Differential Equations
• Generalization bounds for neural ordinary differential equations and deep residual networks