2023/11/09 by Lei Li, Alexander Liniger, Li, Lei +9 · 2 citations
Computer Science · Engineering · #Advanced Memory and Neural Computing #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Robotics (cs.RO) #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.2311.05494
openalex publication_date 2023/11/09 · openalex created_date 2023/11/12 · openalex updated_date 2026/07/28
Event cameras are gaining popularity due to their unique properties, such as their low latency and high dynamic range. One task where these benefits can be crucial is real-time object detection. However, RGB detectors still outperform event-based detectors due to the sparsity of the event data and missing visual details. In this paper, we develop a novel knowledge distillation approach to shrink the performance gap between these two modalities. To this end, we propose a cross-modality object detection distillation method that by design can focus on regions where the knowledge distillation works best. We achieve this by using an object-centric slot attention mechanism that can iteratively decouple features maps into object-centric features and corresponding pixel-features used for distillation. We evaluate our novel distillation approach on a synthetic and a real event dataset with aligned grayscale images as a teacher modality. We show that object-centric distillation allows to significantly improve the performance of the event-based student object detector, nearly halving the performance gap with respect to the teacher.