Learning Efficient Convolutional Networks through Network Slimming

2017/08/22 by Zhuang Liu, Jianguo Li, Liu, Zhuang +9 · 37 citations
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.1708.06519

openalex publication_date 2017/08/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The deployment of deep convolutional neural networks (CNNs) in many real world applications is largely hindered by their high computational cost. In this paper, we propose a novel learning scheme for CNNs to simultaneously 1) reduce the model size; 2) decrease the run-time memory footprint; and 3) lower the number of computing operations, without compromising accuracy. This is achieved by enforcing channel-level sparsity in the network in a simple but effective way. Different from many existing approaches, the proposed method directly applies to modern CNN architectures, introduces minimum overhead to the training process, and requires no special software/hardware accelerators for the resulting models. We call our approach network slimming, which takes wide and large networks as input models, but during training insignificant channels are automatically identified and pruned afterwards, yielding thin and compact models with comparable accuracy. We empirically demonstrate the effectiveness of our approach with several state-of-the-art CNN models, including VGGNet, ResNet and DenseNet, on various image classification datasets. For VGGNet, a multi-pass version of network slimming gives a 20x reduction in model size and a 5x reduction in computing operations.

Citations

• Learning Multiple Layers of Features from Tiny Images

Cited by

• Are the High-weight Neurons the Important Ones in Image Classification Neural Networks?
• Simplifying Multi-Task Architectures Through Task-Specific Normalization
• Seeing the Whole Picture: Distribution-Guided Data-Free Distillation for Semantic Segmentation
• TPV: Parameter Perturbations Through the Lens of Test Prediction Variance
• SHARe-KAN: Post-Training Vector Quantization for Cache-Resident KAN Inference
• Towards Sustainable Universal Deepfake Detection with Frequency-Domain Masking
• HOLE: Homological Observation of Latent Embeddings for Neural Network Interpretability
• Neural expressiveness for beyond importance model compression
• HSCP: A Two-Stage Spectral Clustering Framework for Resource-Constrained UAV Identification
• Extreme Model Compression for Edge Vision-Language Models: Sparse Temporal Token Fusion and Adaptive Neural Compression
• Layer-wise Weight Selection for Power-Efficient Neural Network Acceleration
• Change-of-Basis Pruning via Rotational Invariance
• MI-to-Mid Distilled Compression (M2M-DC): An Hybrid-Information-Guided-Block Pruning with Progressive Inner Slicing Approach to Model Compression
• FastBoost: Progressive Attention with Dynamic Scaling for Efficient Deep Learning
• BN-NAS: Neural Architecture Search with Batch Normalization
• Efficient Perceptual Image Super Resolution: AIM 2025 Study and Benchmark
• Automated Evolutionary Optimization for Resource-Efficient Neural Network Training
• Structured Output Regularization: a framework for few-shot transfer learning
• Train Large, Deploy Compact: Structured Compression for Compact Low-Rank Adaptation
• CAST: Continuous and Differentiable Semi-Structured Sparsity-Aware Training for Large Language Models
• Effective Model Pruning
• Differentiable Sparsity via D-Gating: Simple and Versatile Structured Penalization
• Convolutional Neural Network Optimization for Beehive Classification Using Bioacoustic Signals
• ConceptFlow: Hierarchical and Fine-grained Concept-Based Explanation for Convolutional Neural Networks
• A Novel Compression Framework for YOLOv8: Achieving Real-Time Aerial Object Detection on Edge Devices via Structured Pruning and Channel-Wise Distillation
• Compressing CNN models for resource-constrained systems by channel and layer pruning
• BLK-REW: A Unified Block-based DNN Pruning Framework using Reweighted Regularization Method
• Dynamic Sensitivity Filter Pruning using Multi-Agent Reinforcement Learning For DCNN's
• Resource-Aware Neural Network Pruning Using Graph-based Reinforcement Learning
• A Lightweight Dual-Mode Optimization for Generative Face Video Coding
• Designing Object Detection Models for TinyML: Foundations, Comparative Analysis, Challenges, and Emerging Solutions
• Lightweight Multi-Scale Feature Extraction with Fully Connected LMF Layer for Salient Object Detection
• Optimal Brain Connection: Towards Efficient Structural Pruning
• V.I.P. : Iterative Online Preference Distillation for Efficient Video Diffusion Models
• FAIR-Pruner: Leveraging Tolerance of Difference for Flexible Automatic Layer-Wise Neural Network Pruning
• FedAPTA: Federated Multi-task Learning for Heterogeneous Devices with Adaptive Layer-wise Pruning and Task-aware Aggregation
• Fractional Skipping: Towards Finer-Grained Dynamic CNN Inference

Related

• Improving Network Slimming with Nonconvex Regularization
• Training CNNs faster with Dynamic Input and Kernel Downsampling
• The Power of Sparsity in Convolutional Neural Networks
• Memory Bounded Deep Convolutional Networks
• A Proximal Algorithm for Network Slimming