2024/09/09 by Minsuk Choi, Choi, Minsuk, Hiroshi Ishikawa +1 · 1 citation
Neuroscience · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #EEG and Brain-Computer Interfaces #FOS: Computer and information sciences
paper · pdf · doi:10.48550/arxiv.2409.05279
openalex publication_date 2024/09/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Decoding neural representations of visual stimuli from electroencephalography (EEG) offers valuable insights into brain activity and cognition. Recent advancements in deep learning have significantly enhanced the field of visual decoding of EEG, primarily focusing on reconstructing the semantic content of visual stimuli. In this paper, we present a novel visual decoding pipeline that, in addition to recovering the content, emphasizes the reconstruction of the style, such as color and texture, of images viewed by the subject. Unlike previous methods, this ``style-based'' approach learns in the CLIP spaces of image and text separately, facilitating a more nuanced extraction of information from EEG signals. We also use captions for text alignment simpler than previously employed, which we find work better. Both quantitative and qualitative evaluations show that our method better preserves the style of visual stimuli and extracts more fine-grained semantic information from neural signals. Notably, it achieves significant improvements in quantitative results and sets a new state-of-the-art on the popular Brain2Image dataset.