2023/03/14 by Kaiqi Zhao, A. Jain, Zhao, Kaiqi +3 · 4 citations
Computer Science · #Advanced Neural Network Applications #Computational Physics and Python Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel Computing and Optimization Techniques
paper · pdf · doi:10.48550/arxiv.2303.08595
openalex publication_date 2023/03/14 · openalex created_date 2023/03/18 · openalex updated_date 2026/07/28
Pruning is a promising approach to compress deep learning models in order to deploy them on resource-constrained edge devices. However, many existing pruning solutions are based on unstructured pruning, which yields models that cannot efficiently run on commodity hardware; and they often require users to manually explore and tune the pruning process, which is time-consuming and often leads to sub-optimal results. To address these limitations, this paper presents Automatic Attention Pruning (AAP), an adaptive, attention-based, structured pruning approach to automatically generate small, accurate, and hardware-efficient models that meet user objectives. First, it proposes iterative structured pruning using activation-based attention maps to effectively identify and prune unimportant filters. Then, it proposes adaptive pruning policies for automatically meeting the pruning objectives of accuracy-critical, memory-constrained, and latency-sensitive tasks. A comprehensive evaluation shows that AAP substantially outperforms the state-of-the-art structured pruning works for a variety of model architectures. Our code is at: https://github.com/kaiqi123/Automatic-Attention-Pruning.git.