Neither Quick Nor Proper -- Evaluation of QuickProp for Learning Deep Neural Networks

2016/06/14 by Clemens-Alexander Brust, Sven Sickert, Brust, Clemens-Alexander +7
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #cs.CV

paper · pdf · doi:10.48550/arxiv.1606.04333

Technical Report, 11 pages, 6 figures

openalex publication_date 2016/06/14 · arxiv created 2016/06/15 · arxiv updated 2016/06/16 · openalex created_date 2016/06/24 · openalex updated_date 2026/07/28

Abstract

Neural networks and especially convolutional neural networks are of great interest in current computer vision research. However, many techniques, extensions, and modifications have been published in the past, which are not yet used by current approaches. In this paper, we study the application of a method called QuickProp for training of deep neural networks. In particular, we apply QuickProp during learning and testing of fully convolutional networks for the task of semantic segmentation. We compare QuickProp empirically with gradient descent, which is the current standard method. Experiments suggest that QuickProp can not compete with standard gradient descent techniques for complex computer vision tasks like semantic segmentation.

Citations

• ADADELTA: An Adaptive Learning Rate Method
• Backpropagation Applied to Handwritten Zip Code Recognition
• ImageNet classification with deep convolutional neural networks

Related

• Convolutional Networks as Extremely Small Foundation Models: Visual Prompting and Theoretical Perspective
• Deep Learning For Computer Vision Tasks: A review
• Fast Neural Architecture Search of Compact Semantic Segmentation Models via Auxiliary Cells
• Accelerating Deep Learning by Focusing on the Biggest Losers
• Fully Convolutional Networks for Semantic Segmentation