2024/06/20 by Yang Wang, Haiyang Mei, Wang, Yang +13 · 3 citations
Engineering · Neuroscience · #Advanced Memory and Neural Computing #Computer Vision and Pattern Recognition (cs.CV) #EEG and Brain-Computer Interfaces #FOS: Computer and information sciences #Neural and Evolutionary Computing (cs.NE) #Neural dynamics and brain function
paper · pdf · doi:10.48550/arxiv.2407.09521
openalex publication_date 2024/06/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We introduce a novel multimodality synergistic knowledge distillation scheme tailored for efficient single-eye motion recognition tasks. This method allows a lightweight, unimodal student spiking neural network (SNN) to extract rich knowledge from an event-frame multimodal teacher network. The core strength of this approach is its ability to utilize the ample, coarser temporal cues found in conventional frames for effective emotion recognition. Consequently, our method adeptly interprets both temporal and spatial information from the conventional frame domain, eliminating the need for specialized sensing devices, e.g., event-based camera. The effectiveness of our approach is thoroughly demonstrated using both existing and our compiled single-eye emotion recognition datasets, achieving unparalleled performance in accuracy and efficiency over existing state-of-the-art methods.