vix.ing · top · new · best · stats

Neural Ordinary Differential Equations

2018/06/19 by Ricky T. Q. Chen, Chen, Ricky T. Q., Yulia Rubanova +5 · 5 voices · 978 citations
Computer Science · Mathematics · Physics and Astronomy · #Algorithm #Applied mathematics #Artificial intelligence #Artificial neural network #Black box #Computational Physics and Python Applications #Computer science #Constant (computer programming) #Construct (python library) #Continuous modelling #Differential (mechanical device) #Differential equation #Generative Adversarial Networks and Image Synthesis #Latent variable #Mathematical analysis #Mathematical optimization #Mathematics #Model Reduction and Neural Networks #Ode #Ordinary differential equation #Residual #Sequence (biology) #Solver #Variable (mathematics) #cs.AI #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1806.07366

published in arXiv (Cornell University) 31, 6572-6583 (Cornell University)

openalex publication_date 2018/06/19 · arxiv created 2019/12/14 · arxiv updated 2019/12/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

We introduce a new family of deep neural network models. Instead of specifying a discrete sequence of hidden layers, we parameterize the derivative of the hidden state using a neural network. The output of the network is computed using a black-box differential equation solver. These continuous-depth models have constant memory cost, adapt their evaluation strategy to each input, and can explicitly trade numerical precision for speed. We demonstrate these properties in continuous-depth residual networks and continuous-time latent variable models. We also construct continuous normalizing flows, a generative model that can train by maximum likelihood, without partitioning or ordering the data dimensions. For training, we show how to scalably backpropagate through any ODE solver, without access to its internal operations. This allows end-to-end training of ODEs within larger models.

Citations

Cited by

Discussions

Related