N-ImageNet: Towards Robust, Fine-Grained Object Recognition with Event Cameras

2021/12/02 by Junho Kim, Jaehyeok Bae, Kim, Junho +7 · 20 citations
Engineering · Computer Science · #Advanced Memory and Neural Computing #Ferroelectric and Negative Capacitance Devices #Machine Learning and ELM

paper · pdf · doi:10.48550/arxiv.2112.01041

Abstract

We introduce N-ImageNet, a large-scale dataset targeted for robust, fine-grained object recognition with event cameras. The dataset is collected using programmable hardware in which an event camera consistently moves around a monitor displaying images from ImageNet. N-ImageNet serves as a challenging benchmark for event-based object recognition, due to its large number of classes and samples. We empirically show that pretraining on N-ImageNet improves the performance of event-based classifiers and helps them learn with few labeled data. In addition, we present several variants of N-ImageNet to test the robustness of event-based classifiers under diverse camera trajectories and severe lighting conditions, and propose a novel event representation to alleviate the performance degradation. To the best of our knowledge, we are the first to quantitatively investigate the consequences caused by various environmental conditions on event-based object recognition algorithms. N-ImageNet and its variants are expected to guide practical implementations for deploying event-based object recognition algorithms in the real world.

Citations

• ImageNet classification with deep convolutional neural networks

Cited by

• Neuromorphic Object Detection: An In-Depth Study and Future Directions
• 3D Stack In-Sensor-Computing (3DS-ISC): Accelerating Time-Surface Construction for Neuromorphic Event Cameras
• Reconstruction as a Bridge for Event-Based Visual Question Answering
• CS3D: An Efficient Facial Expression Recognition via Event Vision
• Learning to Remove Lens Flare in Event Camera
• EventBench: Towards Comprehensive Benchmarking of Event-based MLLMs
• I2E: Real-Time Image-to-Event Conversion for High-Performance Spiking Neural Networks
• MoCom: Motion-based Inter-MAV Visual Communication Using Event Vision and Spiking Neural Networks
• ControlEvents: Controllable Synthesis of Event Camera Datawith Foundational Prior from Image Diffusion Models
• OmniSegmentor: A Flexible Multi-Modal Learning Framework for Semantic Segmentation
• Visual Grounding from Event Cameras
• Raw2Event: Converting Raw Frame Camera into Event Camera
• Beyond conventional vision: RGB-event fusion for robust object detection in dynamic traffic scenarios
• LET-US: Long Event-Text Understanding of Scenes
• Revealing Latent Information: A Physics-inspired Self-supervised Pre-training Framework for Noisy and Sparse Events
• evTransFER: A Transfer Learning Framework for Event-based Facial Expression Recognition
• OmniEvent: Unified Event Representation Learning
• Edge Intelligence with Spiking Neural Networks
• Talk2Event: Grounded Understanding of Dynamic Scenes from Event Cameras
• Asynchronous Event Error-Minimizing Noise for Safeguarding Event Dataset

Related

• Learning to Detect Objects with a 1 Megapixel Event Camera
• HATS: Histograms of Averaged Time Surfaces for Robust Event-based Object\n Classification
• EventGAN: Leveraging Large Scale Image Datasets for Event Cameras
• Event-Based Vision: A Survey
• Event Camera Data Pre-training