IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks

2018/05/31 by Ke Sun, Mingjie Li, Sun, Ke +5 · 9 citations
Computer Science · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning and ELM #cs.CV

paper · pdf · doi:10.48550/arxiv.1806.00178

10 pages, 2 figures, accepted by BMVC 2018

openalex publication_date 2018/06/01 · arxiv created 2018/07/20 · arxiv updated 2018/07/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we are interested in building lightweight and efficient convolutional neural networks. Inspired by the success of two design patterns, composition of structured sparse kernels, e.g., interleaved group convolutions (IGC), and composition of low-rank kernels, e.g., bottle-neck modules, we study the combination of such two design patterns, using the composition of structured sparse low-rank kernels, to form a convolutional kernel. Rather than introducing a complementary condition over channels, we introduce a loose complementary condition, which is formulated by imposing the complementary condition over super-channels, to guide the design for generating a dense convolutional kernel. The resulting network is called IGCV3. We empirically demonstrate that the combination of low-rank and sparse kernels boosts the performance and the superiority of our proposed approach to the state-of-the-arts, IGCV2 and MobileNetV2 over image classification on CIFAR and ImageNet and object detection on COCO.

Cited by

• ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
• Interlaced Sparse Self-Attention for Semantic Segmentation
• Shift-and-Balance Attention
• Neural Epitome Search for Architecture-Agnostic Network Compression
• Fully Learnable Group Convolution for Acceleration of Deep Neural Networks
• LPRNet: Lightweight Deep Network by Low-rank Pointwise Residual Convolution
• CondenseNet V2: Sparse Feature Reactivation for Deep Networks
• Dynamic Group Convolution for Accelerating Convolutional Neural Networks
• AutoShuffleNet: Learning Permutation Matrices via an Exact Lipschitz Continuous Penalty in Deep Convolutional Neural Networks

Related

• IGCV2: Interleaved Structured Sparse Convolutional Neural Networks
• Interleaved Group Convolutions for Deep Neural Networks
• clcNet: Improving the Efficiency of Convolutional Neural Network using Channel Local Convolutions
• Boosting Convolutional Neural Networks with Middle Spectrum Grouped Convolution
• Efficient Fusion of Sparse and Complementary Convolutions