2022/03/03 by Shin-ichi Ikegawa, Ryuji Saiin, Ikegawa, Shin-ichi +5 · 2 citations
Computer Science · Engineering · Neuroscience · #Advanced Memory and Neural Computing #Artificial Intelligence (cs.AI) #CCD and CMOS Imaging Sensors #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural and Evolutionary Computing (cs.NE) #Neural dynamics and brain function #cs.AI #cs.CV #cs.LG #cs.NE
paper · pdf · doi:10.48550/arxiv.2203.01544
14 pages, 9 figures, 3 tables
arxiv created 2022/03/03 · openalex publication_date 2022/03/03 · arxiv updated 2022/03/04 · openalex created_date 2022/08/22 · openalex updated_date 2026/07/28
Biologically inspired spiking neural networks (SNNs) are widely used to realize ultralow-power energy consumption. However, deep SNNs are not easy to train due to the excessive firing of spiking neurons in the hidden layers. To tackle this problem, we propose a novel but simple normalization technique called postsynaptic potential normalization. This normalization removes the subtraction term from the standard normalization and uses the second raw moment instead of the variance as the division term. The spike firing can be controlled, enabling the training to proceed appropriating, by conducting this simple normalization to the postsynaptic potential. The experimental results show that SNNs with our normalization outperformed other models using other normalizations. Furthermore, through the pre-activation residual blocks, the proposed model can train with more than 100 layers without other special techniques dedicated to SNNs.