A Sparse CNN Accelerator for Eliminating Redundant Computations in Intra- and Inter-Convolutional/Pooling Layers

2022/11/03 by Chen Yang, Yishuo Meng, Kaibo Huo +2 · 2 citations
Computer Science · Engineering · #Advanced Memory and Neural Computing #Advanced Neural Network Applications #Hand Gesture Recognition Systems

paper · doi:10.1109/tvlsi.2022.3211665

openalex publication_date 2022/11/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/31

Abstract

Neural network pruning, which can be divided into unstructured pruning and structured pruning strategies, has been proven to be an efficient method to substantially reduce the number of computations of convolutional neural networks (CNNs). However, it remains difficult to combine the advantages of these two pruning strategies. This article proposes a high-performance accelerator for unstructured sparse CNNs. First, a convolution-based filter selection and clustering method (FSCM) is proposed to reorder unstructured sparse filters into uniform-size dense filters, eliminating redundant computations in the convolutional layers while maintaining a regular structure. In addition, a convolution and pooling calculation method (CPCM) is presented to reduce interlayers’ computational redundancy. Third, a hardware accelerator with high digital signal processing (DSP) efficiency is designed to take advantage of FSCM and CPCM. The proposed accelerator is implemented on a Xilinx XCVU9P platform at 300 MHz. With different CNN configurations, the performance and DSP efficiency for LeNet, AlexNet, and VGG16 are 1490.53 GOPS and 3.882, 913.85 GOPS and 1.785, and 862.16 GOPS and 1.68, respectively. Compared to previous field-programmable gate array (FPGA) and application-specific integrated circuit (ASIC) designs, the accelerator achieves up to15.43× and7.73× speedup in terms of performance and DSP efficiency.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
• ImageNet classification with deep convolutional neural networks
• Learned Threshold Pruning
• A Multi-Modal Distributed Real-Time IoT System for Urban Traffic Control (Invited Paper)

Cited by

• COMET: Co-Optimization of a CNN Model using Efficient-Hardware OBC Techniques
• An Efficient Sparse CNN Inference Accelerator With Balanced Intra- and Inter-PE Workload

Related

• An Efficient Hardware Accelerator for Structured Sparse Convolutional Neural Networks on FPGAs
• Faster CNNs with Direct Sparse Convolutions and Guided Pruning
• SPEC2: SPECtral SParsE CNN Accelerator on FPGAs
• An Efficient Sparse CNN Inference Accelerator With Balanced Intra- and Inter-PE Workload
• Maximizing CNN Accelerator Efficiency Through Resource Partitioning