SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification

2021/05/31 by Huy Phan, Kaare Mikkelsen, Kaare B. Mikkelsen +5 · 314 citations
Computer Science · Engineering · Neuroscience · Psychology · #Artificial intelligence #Computer science #EEG and Brain-Computer Interfaces #Interpretability #Machine learning #Sleep (system call) #Sleep and Wakefulness Research #Sleep and related disorders #cs.LG #eess.SP

paper · pdf · open access · doi:10.1109/tbme.2022.3147187

published in IEEE Transactions on Biomedical Engineering 69(8), 2456-2467 (Institute of Electrical and Electronics Engineers) · This article has been published in IEEE Transactions on Biomedical Engineering

arxiv created 2022/01/26 · arxiv updated 2022/01/27 · openalex publication_date 2022/01/31 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Background: Black-box skepticism is one of the main hindrances impeding deep-learning-based automatic sleep scoring from being used in clinical environments. Methods: Towards interpretability, this work proposes a sequence-to-sequence sleep-staging model, namely SleepTransformer. It is based on the transformer backbone and offers interpretability of the model's decisions at both the epoch and sequence level. We further propose a simple yet efficient method to quantify uncertainty in the model's decisions. The method, which is based on entropy, can serve as a metric for deferring low-confidence epochs to a human expert for further inspection. Results: Making sense of the transformer's self-attention scores for interpretability, at the epoch level, the attention scores are encoded as a heat map to highlight sleep-relevant features captured from the input EEG signal. At the sequence level, the attention scores are visualized as the influence of different neighboring epochs in an input sequence (i.e. the context) to recognition of a target epoch, mimicking the way manual scoring is done by human experts. Conclusion: Additionally, we demonstrate that SleepTransformer performs on par with existing methods on two databases of different sizes. Significance: Equipped with interpretability and the ability of uncertainty quantification, SleepTransformer holds promise for being integrated into clinical settings.

Citations

• Analysis of a sleep-dependent neuronal feedback loop: the slow-wave microcontinuity of the EEG
• PhysioBank, PhysioToolkit, and PhysioNet
• Towards More Accurate Automatic Sleep Staging via Deep Transfer Learning

Cited by

• How Far Can Wearable-Compatible Signals Go? A Controlled Decomposition of Non-EEG Sleep Staging
• SomnoNet: A Lightweight and Interpretable Framework for Sleep Staging Using Single-Channel EEG
• LGFNet: A CTC-Guided Local-Global Fusion Framework for Single-Channel Sleep Staging
• Sleep Modulation: The Challenge of Transitioning from Open Loop to Closed Loop
• NeuroLingua: A Language-Inspired Hierarchical Framework for Multimodal Sleep Stage Classification Using EEG and EOG
• NAPS: Attention-Based Fusion of Heterogeneous Physiological Signals
• Leveraging Generic Time Series Foundation Models for EEG Classification
• MEASURE: Multi-scale Minimal Sufficient Representation Learning for Domain Generalization in Sleep Staging
• Structure-Preserving Transformers for Sequences of SPD Matrices
• A Systematic Evaluation of Self-Supervised Learning for Label-Efficient Sleep Staging with Wearable EEG
• Introducing Multimodal Paradigm for Learning Sleep Staging PSG via General-Purpose Model
• HealthCAT: An Interpretable Encoder-only Transformer Framework for Health Indicator Prediction and Temporal Interpretation of Wearable Sensor Data
• FetalSleepNet: A Transfer Learning Framework with Spectral Equalisation Domain Adaptation for Fetal Sleep Stage Classification
• Multi-Channel Differential Transformer for Cross-Domain Sleep Stage Classification with Heterogeneous EEG and EOG
• Automatic sleep scoring for real-time monitoring and stimulation in individuals with and without sleep apnea
• CSLP-AE: A Contrastive Split-Latent Permutation Autoencoder Framework for Zero-Shot Electroencephalography Signal Conversion
• AdaBrain-Bench: Benchmarking Brain Foundation Models for Brain-Computer Interface Applications
• SLEEPYLAND: trust begins with fair evaluation of automatic sleep staging models
• Transformer-based EEG Decoding: A Survey
• CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding
• SWINSleepNet: A Hierarchical Context-Aware Framework for Sleep Staging (v2)
• PSG-MAE: Robust Multitask Sleep Event Monitoring using Multichannel PSG Reconstruction and Inter-channel Contrastive Learning
• Deep learning-based algorithm accurately classifies sleep stages in preadolescent children with sleep-disordered breathing symptoms and age-matched controls. [europepmc]
• Current status and prospects of automatic sleep stages scoring: Review. [europepmc]
• Micro SleepNet: efficient deep learning model for mobile terminal real-time sleep staging. [europepmc]
• Looking for a balance between visual and automatic sleep scoring. [europepmc]
• Automatic sleep staging by a hybrid model based on deep 1D-ResNet-SE and LSTM with single-channel raw EEG signals. [europepmc]
• Somnotate: A probabilistic sleep stage classifier for studying vigilance state transitions. [europepmc]
• Bringing At-home Pediatric Sleep Apnea Testing Closer to Reality: A Multi-modal Transformer Approach. [europepmc]
• InsightSleepNet: the interpretable and uncertainty-aware deep learning network for sleep staging using continuous Photoplethysmography. [europepmc]
• The Ultra-Long-Term Sleep study: Design, rationale, data stability and user perspective. [europepmc]
• Bridging AI and Clinical Practice: Integrating Automated Sleep Scoring Algorithm with Uncertainty-Guided Physician Review. [europepmc]
• FlexSleepTransformer: a transformer-based sleep staging model with flexible input channel configurations. [europepmc]
• Multimodal sleep staging network based on obstructive sleep apnea. [europepmc]
• Outlier Handling Strategy of Ensembled-Based Sequential Convolutional Neural Networks for Sleep Stage Classification. [europepmc]
• Explainable vision transformer for automatic visual sleep staging on multimodal PSG signals. [europepmc]
• Ear-EEG sleep monitoring data sets. [europepmc]
• The NERVE-ML (neural engineering reproducibility and validity essentials for machine learning) checklist: ensuring machine learning advances neural engineering . [europepmc]
• A foundational transformer leveraging full night, multichannel sleep study data accurately classifies sleep stages. [europepmc]
• Machine learning classifier solving the problem of sleep stage imbalance between overnight sleep. [europepmc]
• CAISR: achieving human-level performance in automated sleep analysis across all clinical sleep metrics. [europepmc]
• Automated analysis of the American Academy of Sleep Medicine Inter-Scorer Reliability gold-standard polysomnogram dataset. [europepmc]
• SleepMFormer: An Efficient Attention Framework with Contrastive Learning for Single-Channel EEG Sleep Staging. [europepmc]
• Mamba-based deep learning approach for sleep staging on a wireless multimodal wearable system without electroencephalography. [europepmc]
• Robotic Tactile Sensing for Early Detection of Frost-Damaged Citrus Fruits with Pressure-Vibration Multimodal Fusion. [europepmc]
• REST-a deep learning tool for automated mouse sleep stage classification. [europepmc]
• An explainable deep learning approach for sleep staging in sleep apnea patients across all age subgroups from pulse oximetry signals. [europepmc]

Related

• Performance and utility trade-off in interpretable sleep staging
• Personalizing deep learning models for automatic sleep staging
• Transparency in Sleep Staging: Deep Learning Method for EEG Sleep Stage Classification with Model Interpretability
• Transformer-Based Sleep Stage Classification Enhanced by Clinical Information
• NeuroSleepNet: A Multi-Head Self-Attention Based Automatic Sleep Scoring Scheme with Spatial and Multi-Scale Temporal Representation Learning