BRIEF: Backward Reduction of CNNs with Information Flow Analysis

2018/07/16 by Yu‐Hsun Lin, Lin, Yu-Hsun, Chun‐Nan Chou +3
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.1807.05726

openalex publication_date 2018/07/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper proposes BRIEF, a backward reduction algorithm that explores compact CNN-model designs from the information flow perspective. This algorithm can remove substantial non-zero weighting parameters (redundant neural channels) of a network by considering its dynamic behavior, which traditional model-compaction techniques cannot achieve. With the aid of our proposed algorithm, we achieve significant model reduction on ResNet-34 in the ImageNet scale (32.3% reduction), which is 3X better than the previous result (10.8%). Even for highly optimized models such as SqueezeNet and MobileNet, we can achieve additional 10.81% and 37.56% reduction, respectively, with negligible performance degradation.

Citations

• Distilling the Knowledge in a Neural Network
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• ImageNet classification with deep convolutional neural networks
• Binarized Neural Networks
• Opening the Black Box of Deep Neural Networks via Information
• Efficient Processing of Deep Neural Networks: A Tutorial and Survey
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Related

• MBS: Macroblock Scaling for CNN Model Reduction
• ChannelNets: Compact and Efficient Convolutional Neural Networks via Channel-Wise Convolutions
• Compact CNN Structure Learning by Knowledge Distillation
• Smaller Models, Better Generalization
• Learning Efficient Convolutional Networks through Network Slimming