Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions

2017/11/22 by Bichen Wu, Alvin Wan, Wu, Bichen +16 · 1 voice · 12 citations
Computer Science · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #Face recognition and analysis #cs.CV

paper · pdf · doi:10.48550/arxiv.1711.08141

openalex publication_date 2017/11/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Neural networks rely on convolutions to aggregate spatial information. However, spatial convolutions are expensive in terms of model size and computation, both of which grow quadratically with respect to kernel size. In this paper, we present a parameter-free, FLOP-free "shift" operation as an alternative to spatial convolutions. We fuse shifts and point-wise convolutions to construct end-to-end trainable shift-based modules, with a hyperparameter characterizing the tradeoff between accuracy and efficiency. To demonstrate the operation's efficacy, we replace ResNet's 3x3 convolutions with shift-based modules for improved CIFAR10 and CIFAR100 accuracy using 60% fewer parameters; we additionally demonstrate the operation's resilience to parameter reduction on ImageNet, outperforming ResNet family members. We finally show the shift operation's applicability across domains, achieving strong performance with fewer parameters on classification, face verification and style transfer.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Instance Normalization: The Missing Ingredient for Fast Stylization
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Cited by

• HaShiFlex: A High-Throughput Hardened Shifter DNN Accelerator with Fine-Tuning Flexibility
• Neural expressiveness for beyond importance model compression
• Beyond ImageNet: Understanding Cross-Dataset Robustness of Lightweight Vision Models
• Motion Feature Network: Fixed Motion Filter for Action Recognition
• Learning Student Networks via Feature Embedding
• FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search
• Deep Lookup Network
• Packing Sparse Convolutional Neural Networks for Efficient Systolic Array Implementations: Column Combining Under Joint Optimization
• ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptation
• Diagonalwise Refactorization: An Efficient Training Method for Depthwise Convolutions
• Optimization of DNN-based HSI Segmentation FPGA-based SoC for ADS: A Practical Approach
• Efficient Hardware Implementation of Incremental Learning and Inference on Chip

Discussions

• Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions [pdf] [hn, 5 points, 1 comments]

Related

• 4-Connected Shift Residual Networks
• Shift-based Primitives for Efficient Convolutional Neural Networks
• All You Need is a Few Shifts: Designing Efficient Convolutional Neural Networks for Image Classification
• Constructing Fast Network through Deconstruction of Convolution
• Warped Convolutions: Efficient Invariance to Spatial Transformations