Cross-filter compression for CNN inference acceleration

2020/05/18 by Fuyuan Lyu, Lyu, Fuyuan, Shien Zhu +3
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences

paper · pdf · doi:10.48550/arxiv.2005.09034

openalex publication_date 2020/05/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Convolution neural network demonstrates great capability for multiple tasks, such as image classification and many others. However, much resource is required to train a network. Hence much effort has been made to accelerate neural network by reducing precision of weights, activation, and gradient. However, these filter-wise quantification methods exist a natural upper limit, caused by the size of the kernel. Meanwhile, with the popularity of small kernel, the natural limit further decrease. To address this issue, we propose a new cross-filter compression method that can provide ∼32× memory savings and 122× speed up in convolution operations. In our method, all convolution filters are quantized to given bits and spatially adjacent filters share the same scaling factor. Our compression method, based on Binary-Weight and XNOR-Net separately, is evaluated on CIFAR-10 and ImageNet dataset with widely used network structures, such as ResNet and VGG, and witness tolerable accuracy loss compared to state-of-the-art quantification methods.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• A guide to convolution arithmetic for deep learning
• Ternary Weight Networks
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Trained Ternary Quantization
• Efficient Processing of Deep Neural Networks: A Tutorial and Survey
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Performance Guaranteed Network Acceleration via High-Order Residual Quantization
• Deep Residual Learning for Image Recognition
• ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
• DeepRebirth: Accelerating Deep Neural Network Execution on Mobile Devices
• Learning Multiple Layers of Features from Tiny Images

Related

• Coreset-Based Neural Network Compression
• FSNet: Compression of Deep Convolutional Neural Networks by Filter Summary
• Recent Advances in Convolutional Neural Network Acceleration
• Compressing Deep CNNs using Basis Representation and Spectral Fine-tuning
• Learning Filter Basis for Convolutional Neural Network Compression