Sleep Stage Classification using Multimodal Embedding Fusion from EOG and PSM

2025/06/07 by Olivier Papillon, Papillon, Olivier, Rafik Goubran +11
Medicine · Neuroscience · #Computer Vision and Pattern Recognition (cs.CV) #EEG and Brain-Computer Interfaces #FOS: Computer and information sciences #Obstructive Sleep Apnea Research #Sleep and Wakefulness Research

paper · pdf · doi:10.48550/arxiv.2506.06912

openalex publication_date 2025/06/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Accurate sleep stage classification is essential for diagnosing sleep disorders, particularly in aging populations. While traditional polysomnography (PSG) relies on electroencephalography (EEG) as the gold standard, its complexity and need for specialized equipment make home-based sleep monitoring challenging. To address this limitation, we investigate the use of electrooculography (EOG) and pressure-sensitive mats (PSM) as less obtrusive alternatives for five-stage sleep-wake classification. This study introduces a novel approach that leverages ImageBind, a multimodal embedding deep learning model, to integrate PSM data with dual-channel EOG signals for sleep stage classification. Our method is the first reported approach that fuses PSM and EOG data for sleep stage classification with ImageBind. Our results demonstrate that fine-tuning ImageBind significantly improves classification accuracy, outperforming existing models based on single-channel EOG (DeepSleepNet), exclusively PSM data (ViViT), and other multimodal deep learning approaches (MBT). Notably, the model also achieved strong performance without fine-tuning, highlighting its adaptability to specific tasks with limited labeled data, making it particularly advantageous for medical applications. We evaluated our method using 85 nights of patient recordings from a sleep clinic. Our findings suggest that pre-trained multimodal embedding models, even those originally developed for non-medical domains, can be effectively adapted for sleep staging, with accuracies approaching systems that require complex EEG data.

Citations

• A review of deep learning-based information fusion techniques for multimodal medical image classification
• Semi-Supervised Disease Classification based on Limited Medical Image Data
• Enhancing Healthcare with EOG: A Novel Approach to Sleep Stage Classification
• ImageBind: One Embedding Space To Bind Them All
• Attention Bottlenecks for Multimodal Fusion
• An Attention-Based Deep Learning Approach for Sleep Stage Classification With Single-Channel EEG
• ImageNet-21K Pretraining for the Masses
• ViViT: A Video Vision Transformer
• The Kinetics Human Action Video Dataset
• Analysis of a sleep-dependent neuronal feedback loop: the slow-wave microcontinuity of the EEG

Related

• Robust Sleep Stage Classification from Multimodal Physiological Signals Using a Cross-Modal Transformer with Missing Data Resilience
• Enhancing Healthcare with EOG: A Novel Approach to Sleep Stage Classification
• A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series
• Toward Foundational Model for Sleep Analysis Using a Multimodal Hybrid Self-Supervised Learning Framework
• Introducing Multimodal Paradigm for Learning Sleep Staging PSG via General-Purpose Model