2022/10/01 by Qingyuan Li, Bo Zhang, Li, Qingyuan +3
Computer Science · Engineering · #Advanced Memory and Neural Computing #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Neural Networks and Applications #cs.CV
paper · pdf · doi:10.48550/arxiv.2210.00181
BMVC 2022
arxiv created 2022/10/01 · openalex publication_date 2022/10/01 · arxiv updated 2022/10/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Structured pruning greatly eases the deployment of large neural networks in resource-constrained environments. However, current methods either involve strong domain expertise, require extra hyperparameter tuning, or are restricted only to a specific type of network, which prevents pervasive industrial applications. In this paper, we undertake a simple and effective approach that can be easily applied to both vision transformers and convolutional neural networks. Specifically, we consider pruning as an evolution process of sub-network structures that inherit weights through reconstruction techniques. We achieve a 50% FLOPS reduction for ResNet50 and MobileNetV1, leading to 1.37x and 1.34x speedup respectively. For DeiT-Base, we reach nearly 40% FLOPs reduction and 1.4x speedup. Our code will be made available.