Representation Capacity-Matched QNN-SNN Twin Construction for Rate-Encoded SNNs

2024/08/29 by Zhanglu Yan, Yan, Zhanglu, Zhenyu Bai +5 · 6 citations
Computer Science · Engineering · #Advanced Memory and Neural Computing #Neural Networks and Applications #cs.AI #cs.LG #cs.NE

paper · pdf · doi:10.48550/arxiv.2409.08290

openalex publication_date 2024/08/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/31

Abstract

Spiking Neural Networks (SNNs) promise higher energy efficiency over conventional Quantized Artificial Neural Networks (QNNs) due to their event-driven, spike-based computation. However, prevailing energy evaluations often oversimplify, focusing on computational aspects while neglecting critical overheads like comprehensive data movements and memory accesses. Such simplifications can lead to misleading conclusions regarding the true energy benefits of SNNs. This paper presents a rigorous re-evaluation. We establish a fair baseline by mapping rate-encoded SNNs with T timesteps to capacity-matched QNNs with \lceil log₂(T+1) \rceil bits. This ensures both models have comparable representational capacities, as well as similar hardware requirements, enabling meaningful energy comparisons. We introduce a detailed analytical energy model encompassing core computation and data movements. Using this model, we systematically explore a wide parameter space, including intrinsic network characteristics (SNN time window size, spike rate, QNN sparsity, model size, weight bit-level) and hardware characteristics (memory system and network-on-chip). Our analysis identifies specific operational regimes where SNNs genuinely offer superior energy efficiency. For example, under typical neuromorphic hardware conditions, SNNs with moderate time windows (T = 5) require an average spike rate (s_r) below 5.7% to outperform equivalent QNNs These insights guide the design of truly energy-efficient neural network solutions.

Cited by

• From LIF to QIF: Toward Differentiable Spiking Neurons for Scientific Machine Learning
• Spiking Neural Networks: The Future of Brain-Inspired Computing
• Energy-Efficient and Dequantization-Free Q-LLMs: A Spiking Neural Network Approach to Salient Value Mitigation
• Random Feature Spiking Neural Networks
• Otters: An Energy-Efficient SpikingTransformer via Optical Time-to-First-Spike Encoding
• Hardware-aware vs. Hardware-agnostic Energy Estimation for SNN in Space Applications

Related

• Is Conventional SNN Really Efficient? A Perspective from Network Quantization
• Are SNNs Truly Energy-efficient? - A Hardware Perspective
• You Only Spike Once: Improving Energy-Efficient Neuromorphic Inference to ANN-Level Accuracy
• To Spike or Not to Spike? A Quantitative Comparison of SNN and CNN FPGA Implementations
• A Unified Evaluation Framework for Spiking Neural Network Hardware Accelerators Based on Emerging Non-Volatile Memory Devices