2018/04/05 by Xin Yu, Yu, Xin, Zhiding Yu +3
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #cs.CV
paper · pdf · doi:10.48550/arxiv.1804.01661
Make title consistent with the CVPR version
openalex publication_date 2018/04/05 · arxiv created 2019/06/16 · arxiv updated 2019/06/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
A family of super deep networks, referred to as residual networks or ResNet, achieved record-beating performance in various visual tasks such as image recognition, object detection, and semantic segmentation. The ability to train very deep networks naturally pushed the researchers to use enormous resources to achieve the best performance. Consequently, in many applications super deep residual networks were employed for just a marginal improvement in performance. In this paper, we propose epsilon-ResNet that allows us to automatically discard redundant layers, which produces responses that are smaller than a threshold epsilon, with a marginal or no loss in performance. The epsilon-ResNet architecture can be achieved using a few additional rectified linear units in the original ResNet. Our method does not use any additional variables nor numerous trials like other hyper-parameter optimization techniques. The layer selection is achieved using a single training process and the evaluation is performed on CIFAR-10, CIFAR-100, SVHN, and ImageNet datasets. In some instances, we achieve about 80% reduction in the number of parameters.