Transfer Learning using Neural Ordinary Differential Equations

2020/01/21 by S, Rajath, K, Sumukh Aithal, Subramanyam, Natarajan
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2001.07342

Abstract

A concept of using Neural Ordinary Differential Equations(NODE) for Transfer Learning has been introduced. In this paper we use the EfficientNets to explore transfer learning on CIFAR-10 dataset. We use NODE for fine-tuning our model. Using NODE for fine tuning provides more stability during training and validation.These continuous depth blocks can also have a trade off between numerical precision and speed .Using Neural ODEs for transfer learning has resulted in much stable convergence of the loss function.

Related

• TransNet: Transferable Neural Networks for Partial Differential Equations
• Physics-Informed Neural Networks for High-Frequency and Multi-Scale Problems using Transfer Learning
• Learning the Simplest Neural ODE
• Convolutional-neural-operator-based transfer learning for solving PDEs
• One-Shot Transfer Learning of Physics-Informed Neural Networks