Optimizing the Consumption of Spiking Neural Networks with Activity Regularization

2022/04/04 by Simon Narduzzi, Narduzzi, Simon, Siavash Bigdeli +5
Engineering · Neuroscience · #Advanced Memory and Neural Computing #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Neural and Evolutionary Computing (cs.NE) #Neural dynamics and brain function

paper · pdf · doi:10.48550/arxiv.2204.01460

openalex publication_date 2022/04/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Reducing energy consumption is a critical point for neural network models running on edge devices. In this regard, reducing the number of multiply-accumulate (MAC) operations of Deep Neural Networks (DNNs) running on edge hardware accelerators will reduce the energy consumption during inference. Spiking Neural Networks (SNNs) are an example of bio-inspired techniques that can further save energy by using binary activations, and avoid consuming energy when not spiking. The networks can be configured for equivalent accuracy on a task through DNN-to-SNN conversion frameworks but their conversion is based on rate coding therefore the synaptic operations can be high. In this work, we look into different techniques to enforce sparsity on the neural network activation maps and compare the effect of different training regularizers on the efficiency of the optimized DNNs and SNNs.

Related

• On Reducing Activity with Distillation and Regularization for Energy Efficient Spiking Neural Networks
• Optimizing the energy consumption of spiking neural networks for\n neuromorphic applications
• When Bio-Inspired Computing meets Deep Learning: Low-Latency, Accurate, & Energy-Efficient Spiking Neural Networks from Artificial Neural Networks
• SATA: Sparsity-Aware Training Accelerator for Spiking Neural Networks
• Can Deep Neural Networks be Converted to Ultra Low-Latency Spiking Neural Networks?