Learning from Event Cameras with Sparse Spiking Convolutional Neural\n Networks

2021/04/26 by Loïc Cordone, Cordone, Loïc, Benoît Miramond +3 · 4 citations
Engineering · Neuroscience · #Advanced Memory and Neural Computing #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Neural dynamics and brain function

paper · pdf · doi:10.48550/arxiv.2104.12579

openalex publication_date 2021/04/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Convolutional neural networks (CNNs) are now the de facto solution for\ncomputer vision problems thanks to their impressive results and ease of\nlearning. These networks are composed of layers of connected units called\nartificial neurons, loosely modeling the neurons in a biological brain.\nHowever, their implementation on conventional hardware (CPU/GPU) results in\nhigh power consumption, making their integration on embedded systems difficult.\nIn a car for example, embedded algorithms have very high constraints in term of\nenergy, latency and accuracy. To design more efficient computer vision\nalgorithms, we propose to follow an end-to-end biologically inspired approach\nusing event cameras and spiking neural networks (SNNs). Event cameras output\nasynchronous and sparse events, providing an incredibly efficient data source,\nbut processing these events with synchronous and dense algorithms such as CNNs\ndoes not yield any significant benefits. To address this limitation, we use\nspiking neural networks (SNNs), which are more biologically realistic neural\nnetworks where units communicate using discrete spikes. Due to the nature of\ntheir operations, they are hardware friendly and energy-efficient, but training\nthem still remains a challenge. Our method enables the training of sparse\nspiking convolutional neural networks directly on event data, using the popular\ndeep learning framework PyTorch. The performances in terms of accuracy,\nsparsity and training time on the popular DVS128 Gesture Dataset make it\npossible to use this bio-inspired approach for the future embedding of\nreal-time applications on low-power neuromorphic hardware.\n

Cited by

• Benchmarking Microsaccade Recognition with Event Cameras: A Novel Dataset and Evaluation
• Incorporating Visual Cortical Lateral Connection Properties into CNN: Recurrent Activation and Excitatory-Inhibitory Separation
• Spike-TBR: a Noise Resilient Neuromorphic Event Representation
• Self-Supervised Event Representations: Towards Accurate, Real-Time Perception on SoC FPGAs

Related

• Learning from Dense Events: Towards Fast Spiking Neural Networks Training via Event Dataset Distillation
• Event-based Spiking Neural Networks for Object Detection: A Review of Datasets, Architectures, Learning Rules, and Implementation
• SpikePoint: An Efficient Point-based Spiking Neural Network for Event Cameras Action Recognition
• Razor SNN: Efficient Spiking Neural Network with Temporal Embeddings
• Going Deeper in Spiking Neural Networks: VGG and Residual Architectures