2021/04/26 by Sayeed Shafayet Chowdhury, Chowdhury, Sayeed Shafayet, Isha Garg +3 · 4 citations
Computer Science · Engineering · Neuroscience · #Advanced Memory and Neural Computing #FOS: Computer and information sciences #FOS: Electrical engineering #Image and Video Processing (eess.IV) #Machine Learning (cs.LG) #Neural Networks and Reservoir Computing #Neural dynamics and brain function #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2104.12528
openalex publication_date 2021/04/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Spiking Neural Networks (SNNs) are a promising alternative to traditional\ndeep learning methods since they perform event-driven information processing.\nHowever, a major drawback of SNNs is high inference latency. The efficiency of\nSNNs could be enhanced using compression methods such as pruning and\nquantization. Notably, SNNs, unlike their non-spiking counterparts, consist of\na temporal dimension, the compression of which can lead to latency reduction.\nIn this paper, we propose spatial and temporal pruning of SNNs. First,\nstructured spatial pruning is performed by determining the layer-wise\nsignificant dimensions using principal component analysis of the average\naccumulated membrane potential of the neurons. This step leads to 10-14X model\ncompression. Additionally, it enables inference with lower latency and\ndecreases the spike count per inference. To further reduce latency, temporal\npruning is performed by gradually reducing the timesteps while training. The\nnetworks are trained using surrogate gradient descent based backpropagation and\nwe validate the results on CIFAR10 and CIFAR100, using VGG architectures. The\nspatiotemporally pruned SNNs achieve 89.04% and 66.4% accuracy on CIFAR10 and\nCIFAR100, respectively, while performing inference with 3-30X reduced latency\ncompared to state-of-the-art SNNs. Moreover, they require 8-14X lesser compute\nenergy compared to their unpruned standard deep learning counterparts. The\nenergy numbers are obtained by multiplying the number of operations with energy\nper operation. These SNNs also provide 1-4% higher robustness against Gaussian\nnoise corrupted inputs. Furthermore, we perform weight quantization and find\nthat performance remains reasonably stable up to 5-bit quantization.\n