Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects

2023/06/16 by Kexin Zhang, Zhang, Kexin, Qingsong Wen +19 · 18 citations
Computer Science · #Anomaly Detection Techniques and Applications #Applications (stat.AP) #Artificial Intelligence (cs.AI) #Data Stream Mining Techniques #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Signal Processing (eess.SP) #Time Series Analysis and Forecasting #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2306.10125

openalex publication_date 2023/06/16 · openalex created_date 2023/06/22 · openalex updated_date 2026/07/28

Abstract

Self-supervised learning (SSL) has recently achieved impressive performance on various time series tasks. The most prominent advantage of SSL is that it reduces the dependence on labeled data. Based on the pre-training and fine-tuning strategy, even a small amount of labeled data can achieve high performance. Compared with many published self-supervised surveys on computer vision and natural language processing, a comprehensive survey for time series SSL is still missing. To fill this gap, we review current state-of-the-art SSL methods for time series data in this article. To this end, we first comprehensively review existing surveys related to SSL and time series, and then provide a new taxonomy of existing time series SSL methods by summarizing them from three perspectives: generative-based, contrastive-based, and adversarial-based. These methods are further divided into ten subcategories with detailed reviews and discussions about their key intuitions, main frameworks, advantages and disadvantages. To facilitate the experiments and validation of time series SSL methods, we also summarize datasets commonly used in time series forecasting, classification, anomaly detection, and clustering tasks. Finally, we present the future directions of SSL for time series analysis.

Cited by

• Concurrence: A dependence criterion for time series, applied to biological data
• QuanvNeXt: An end-to-end quanvolutional neural network for EEG-based detection of major depressive disorder
• DK-Root: A Joint Data-and-Knowledge-Driven Framework for Root Cause Analysis of QoE Degradations in Mobile Networks
• FMMVCC: Fuzzy Mamba-based Multi-View Contrastive Clustering for Univariate Time Series
• Exploring Structural Degradation in Dense Representations for Self-supervised Learning
• Unsupervised Transformer Pre-Training for Images: Self-Distillation, Mean Teachers, and Random Crops
• In-Situ Fault Detection of Submerged Pump Impellers Using Encapsulated Accelerometers and Machine Learning
• Detecting Multilevel Manipulation from Limit Order Book via Cascaded Contrastive Representation Learning
• PLanTS: Periodicity-aware Latent-state Representation Learning for Multivariate Time Series
• CCE: Confidence-Consistency Evaluation for Time Series Anomaly Detection
• A Unified Contrastive-Generative Framework for Time Series Classification
• Discrepancy-Aware Contrastive Adaptation in Medical Time Series Analysis
• Data Cleaning of Data Streams
• Towards a general-purpose foundation model for fMRI analysis
• Reconciling Attribute and Structural Anomalies for Improved Graph Anomaly Detection
• Multi-View Contrastive Learning for Robust Domain Adaptation in Medical Time Series Analysis
• Scaling Transformers for Time Series Forecasting: Do Pretrained Large Models Outperform Small-Scale Alternatives?
• CoIFNet: A Unified Framework for Multivariate Time Series Forecasting with Missing Values

Related

• Self-Supervised Learning for Time Series: Contrastive or Generative?
• A Unified Contrastive-Generative Framework for Time Series Classification
• A Review on Self-Supervised Learning for Time Series Anomaly Detection: Recent Advances and Open Challenges
• Self-Distilled Representation Learning for Time Series
• TimeCSL: Unsupervised Contrastive Learning of General Shapelets for Explorable Time Series Analysis