2021/05/13 by Mehrdad Maleki, Mansura Habiba, Maleki, Mehrdad +3
Computer Science · Physics and Astronomy · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Neural and Evolutionary Computing (cs.NE)
paper · pdf · doi:10.48550/arxiv.2105.06168
openalex publication_date 2021/05/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
There is an analogy between the ResNet (Residual Network) architecture for deep neural networks and an Euler solver for an ODE. The transformation performed by each layer resembles an Euler step in solving an ODE. We consider the Heun Method, which involves a single predictor-corrector cycle, and complete the analogy, building a predictor-corrector variant of ResNet, which we call a HeunNet. Just as Heun's method is more accurate than Euler's, experiments show that HeunNet achieves high accuracy with low computational (both training and test) time compared to both vanilla recurrent neural networks and other ResNet variants.