MAEEG: Masked Auto-encoder for EEG Representation Learning

2022/10/27 by Hsiang-Yun Sherry Chien, Chien, Hsiang-Yun Sherry, Hanlin Goh +5 · 15 citations
Computer Science · Neuroscience · #Blind Source Separation Techniques #EEG and Brain-Computer Interfaces #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Neural dynamics and brain function #Signal Processing (eess.SP) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2211.02625

openalex publication_date 2022/10/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/31

Abstract

Decoding information from bio-signals such as EEG, using machine learning has been a challenge due to the small data-sets and difficulty to obtain labels. We propose a reconstruction-based self-supervised learning model, the masked auto-encoder for EEG (MAEEG), for learning EEG representations by learning to reconstruct the masked EEG features using a transformer architecture. We found that MAEEG can learn representations that significantly improve sleep stage classification (~5% accuracy increase) when only a small number of labels are given. We also found that input sample lengths and different ways of masking during reconstruction-based SSL pretraining have a huge effect on downstream model performance. Specifically, learning to reconstruct a larger proportion and more concentrated masked signal results in better performance on sleep classification. Our findings provide insight into how reconstruction-based SSL could help representation learning for EEG.

Cited by

• SPECTRE: Spectral Pre-training Embeddings with Cylindrical Temporal Rotary Position Encoding for Fine-Grained sEMG-Based Movement Decoding
• Neonatal Hypoxic-ischaemic Encephalopathy Classification from the EEG and HRV Signals Using a Conformer based Masked Autoencoder
• Learning Time-Scale Invariant Population-Level Neural Representations
• SAMBA: Toward a Long-Context EEG Foundation Model via Spatial Embedding and Differential Mamba
• Learning the relative composition of EEG signals using pairwise relative shift pretraining
• REVE: A Foundation Model for EEG -- Adapting to Any Setup with Large-Scale Pretraining on 25,000 Subjects
• A Systematic Evaluation of Self-Supervised Learning for Label-Efficient Sleep Staging with Wearable EEG
• EEGDM: Learning EEG Representation with Latent Diffusion Model
• Masked Autoencoders for Ultrasound Signals: Robust Representation Learning for Downstream Applications
• EEGDM: EEG Representation Learning via Generative Diffusion Model
• A Unified Contrastive-Generative Framework for Time Series Classification
• Never compromise with vulnerabilities: a comprehensive survey on AI governance
• Flow Matching Meets Biology and Life Science: A Survey
• Transformer-based EEG Decoding: A Survey
• Global and Local Contrastive Learning for Joint Representations from Cardiac MRI and ECG

Related

• Self-supervised Contrastive Learning for EEG-based Sleep Staging
• Self-supervised EEG Representation Learning for Automatic Sleep Staging
• Uncovering the structure of clinical EEG signals with self-supervised\n learning
• BENDR: using transformers and a contrastive self-supervised learning\n task to learn from massive amounts of EEG data
• Learning the relative composition of EEG signals using pairwise relative shift pretraining