2018/05/16 by Vinay Jayaram, Alexandre Barachant · 1 citation
Computer Science · Neuroscience · #Adversarial Robustness in Machine Learning #Benchmarking #Brain–computer interface #EEG and Brain-Computer Interfaces #Explainable Artificial Intelligence (XAI) #Measure (data warehouse) #Pattern recognition (psychology) #Trustworthiness #cs.HC
paper · pdf · doi:10.1088/1741-2552/aadea0
arxiv created 2018/05/16 · openalex created_date 2018/06/01 · openalex publication_date 2018/09/04 · arxiv updated 2018/09/11 · openalex updated_date 2026/08/05
OBJECTIVE: Brain-computer interface (BCI) algorithm development has long been hampered by two major issues: small sample sets and a lack of reproducibility. We offer a solution to both of these problems via a software suite that streamlines both the issues of finding and preprocessing data in a reliable manner, as well as that of using a consistent interface for machine learning methods. APPROACH: By building on recent advances in software for signal analysis implemented in the MNE toolkit, and the unified framework for machine learning offered by the scikit-learn project, we offer a system that can improve BCI algorithm development. This system is fully open-source under the BSD licence and available at https://github.com/NeuroTechX/moabb. MAIN RESULTS: We analyze a set of state-of-the-art decoding algorithms across 12 open access datasets, including over 250 subjects. Our results show that even for the best methods, there are datasets which do not show significant improvements, and further that many previously validated methods do not generalize well outside the datasets they were tested on. SIGNIFICANCE: Our analysis confirms that BCI algorithms validated on single datasets are not representative, highlighting the need for more robust validation in the machine learning for BCIs community.