2020/06/09 by Lukas Muttenthaler, Muttenthaler, Lukas, Nora Hollenstein +3
Computer Science · Neuroscience · #Computation and Language (cs.CL) #EEG and Brain-Computer Interfaces #FOS: Biological sciences #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications #Neurons and Cognition (q-bio.NC) #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2006.05113
openalex publication_date 2020/06/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Cognitively inspired NLP leverages human-derived data to teach machines about language processing mechanisms. Recently, neural networks have been augmented with behavioral data to solve a range of NLP tasks spanning syntax and semantics. We are the first to exploit neuroscientific data, namely electroencephalography (EEG), to inform a neural attention model about language processing of the human brain. The challenge in working with EEG data is that features are exceptionally rich and need extensive pre-processing to isolate signals specific to text processing. We devise a method for finding such EEG features to supervise machine attention through combining theoretically motivated cropping with random forest tree splits. After this dimensionality reduction, the pre-processed EEG features are capable of distinguishing two reading tasks retrieved from a publicly available EEG corpus. We apply these features to regularise attention on relation classification and show that EEG is more informative than strong baselines. This improvement depends on both the cognitive load of the task and the EEG frequency domain. Hence, informing neural attention models with EEG signals is beneficial but requires further investigation to understand which dimensions are the most useful across NLP tasks.