Integrating Sequence and Image Modeling in Irregular Medical Time Series Through Self-Supervised Learning

2025/02/10 by Liuqing Chen, Shuhong Xiao, Chen, Liuqing +7
Computer Science · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning in Healthcare

paper · pdf · doi:10.48550/arxiv.2502.06134

openalex publication_date 2025/02/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Medical time series are often irregular and face significant missingness, posing challenges for data analysis and clinical decision-making. Existing methods typically adopt a single modeling perspective, either treating series data as sequences or transforming them into image representations for further classification. In this paper, we propose a joint learning framework that incorporates both sequence and image representations. We also design three self-supervised learning strategies to facilitate the fusion of sequence and image representations, capturing a more generalizable joint representation. The results indicate that our approach outperforms seven other state-of-the-art models in three representative real-world clinical datasets. We further validate our approach by simulating two major types of real-world missingness through leave-sensors-out and leave-samples-out techniques. The results demonstrate that our approach is more robust and significantly surpasses other baselines in terms of classification performance.

Related

• Sequential Multi-Dimensional Self-Supervised Learning for Clinical Time Series
• Symbol-Temporal Consistency Self-supervised Learning for Robust Time Series Classification
• Time Series as Images: Vision Transformer for Irregularly Sampled Time Series
• Improving Clinical Predictions through Unsupervised Time Series Representation Learning
• TS-P²CL: Plug-and-Play Dual Contrastive Learning for Vision-Guided Medical Time Series Classification