TS2Vec: Towards Universal Representation of Time Series

2021/06/19 by Zhihan Yue, Yujing Wang, Yue, Zhihan +11 · 70 citations
Computer Science · Economics, Econometrics and Finance · #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #Complex Systems and Time Series Analysis #FOS: Computer and information sciences #Machine Learning (cs.LG) #Time Series Analysis and Forecasting

paper · pdf · doi:10.48550/arxiv.2106.10466

openalex publication_date 2021/06/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper presents TS2Vec, a universal framework for learning representations of time series in an arbitrary semantic level. Unlike existing methods, TS2Vec performs contrastive learning in a hierarchical way over augmented context views, which enables a robust contextual representation for each timestamp. Furthermore, to obtain the representation of an arbitrary sub-sequence in the time series, we can apply a simple aggregation over the representations of corresponding timestamps. We conduct extensive experiments on time series classification tasks to evaluate the quality of time series representations. As a result, TS2Vec achieves significant improvement over existing SOTAs of unsupervised time series representation on 125 UCR datasets and 29 UEA datasets. The learned timestamp-level representations also achieve superior results in time series forecasting and anomaly detection tasks. A linear regression trained on top of the learned representations outperforms previous SOTAs of time series forecasting. Furthermore, we present a simple way to apply the learned representations for unsupervised anomaly detection, which establishes SOTA results in the literature. The source code is publicly available at https://github.com/yuezhihan/ts2vec.

Cited by

• Failures Reveal What Metrics Miss: An Evidence-Driven Agent for Recursive Refinement of ECG Classifiers
• EEGForceFusion: Joint Tokenised-Continuous Representation Learning for Subject-Independent Grasp Force Decoding
• DriveDNA: A Large-Scale Multimodal Naturalistic Driving Dataset and Benchmark for Driving Style Identification
• TF-MCL: Time-frequency Fusion and Multi-domain Cross-Loss for Self-supervised Depression Detection
• BaRISTA: Brain Scale Informed Spatiotemporal Representation of Human Intracranial Neural Activity
• Contrastive Time Series Forecasting with Anomalies
• Weighted Contrastive Learning for Anomaly-Aware Time-Series Forecasting
• A Self-explainable Model of Long Time Series by Extracting Informative Structured Causal Patterns
• Self-Supervised Dynamical System Representations for Physiological Time-Series
• TS2Vec-Ensemble: An Enhanced Self-Supervised Framework for Time Series Forecasting
• Learning Time-Scale Invariant Population-Level Neural Representations
• A Problem-Oriented Taxonomy of Evaluation Metrics for Time Series Anomaly Detection
• Mask the Redundancy: Evolving Masking Representation Learning for Multivariate Time-Series Clustering
• Lightweight Foundation Model for Wireless Time Series Downstream Tasks on Edge Devices
• Dynamic Reward Scaling for Multivariate Time Series Anomaly Detection: A VAE-Enhanced Reinforcement Learning Approach
• Beyond MSE: Ordinal Cross-Entropy for Probabilistic Time Series Forecasting
• DK-Root: A Joint Data-and-Knowledge-Driven Framework for Root Cause Analysis of QoE Degradations in Mobile Networks
• Koopman Invariants as Drivers of Emergent Time-Series Clustering in Joint-Embedding Predictive Architectures
• One Model for All: Universal Pre-training for EEG based Emotion Recognition across Heterogeneous Datasets and Paradigms
• COGNOS: Universal Enhancement for Time Series Anomaly Detection via Constrained Gaussian-Noise Optimization and Smoothing
• FAPEX: Fractional Amplitude-Phase Expressor for Robust Cross-Subject Seizure Prediction
• Pre-trained Forecasting Models: Strong Zero-Shot Feature Extractors for Time Series Classification
• Contrastive Representation Learning of Longitudinal Disease Trajectories on Temporal Graphs
• FMMVCC: Fuzzy Mamba-based Multi-View Contrastive Clustering for Univariate Time Series
• Sequence Modeling with Spectral Mean Flows
• Learning Without Augmenting: Unsupervised Time Series Representation Learning via Frame Projections
• Flight Delay Prediction via Cross-Modality Adaptation of Large Language Models and Aircraft Trajectory Representation
• Abstain Mask Retain Core: Time Series Prediction by Adaptive Masking Loss with Representation Consistency
• Online Kernel Dynamic Mode Decomposition for Streaming Time Series Forecasting with Adaptive Windowing
• Generalist vs Specialist Time Series Foundation Models: Investigating Potential Emergent Behaviors in Assessing Human Health Using PPG Signals
• Leaking Queries On Secure Stream Processing Systems
• STaTS: Structure-Aware Temporal Sequence Summarization via Statistical Window Merging
• Learning Time-Series Representations by Hierarchical Uniformity-Tolerance Latent Balancing
• A Diffusion Model for Regular Time Series Generation from Irregular Data with Completion and Masking
• THEMIS: Unlocking Pretrained Knowledge with Foundation Model Embeddings for Anomaly Detection in Time Series
• Joint Embeddings Go Temporal
• IndexNet: Timestamp and Variable-Aware Modeling for Time Series Forecasting
• Conditionally Whitened Generative Models for Probabilistic Time Series Forecasting
• Manifold-Aware Diffusion-Augmented Contrastive Learning for Noise-Robust Biosignal Representation
• Symbol-Temporal Consistency Self-supervised Learning for Robust Time Series Classification
• OSF: On Pre-training and Scaling of Sleep Foundation Models
• Conv-like Scale-Fusion Time Series Transformer: A Multi-Scale Representation for Variable-Length Long Time Series
• MTS-DMAE: Dual-Masked Autoencoder for Unsupervised Multivariate Time Series Representation Learning
• DeCoP: Enhancing Self-Supervised Time Series Representation with Dependency Controlled Pre-training
• Bridging Past and Future: Distribution-Aware Alignment for Time Series Forecasting
• Water Demand Forecasting of District Metered Areas through Learned Consumer Representations
• General Demographic Foundation Models for Enhancing Predictive Performance Across Diseases and Populations
• PLanTS: Periodicity-aware Latent-state Representation Learning for Multivariate Time Series
• Privacy Risks in Time Series Forecasting: User- and Record-Level Membership Inference
• One-Embedding-Fits-All: Efficient Zero-Shot Time Series Forecasting by a Model Zoo
• Insight-LLM: LLM-enhanced Multi-view Fusion in Insider Threat Detection
• Decomposing and Revising What Language Models Generate
• Cross-device Zero-shot Label Transfer via Alignment of Time Series Foundation Model Embeddings
• Human-AI Collaborative Bot Detection in MMORPGs
• Borrowing From the Future: Enhancing Early Risk Assessment through Contrastive Learning
• A Unified Contrastive-Generative Framework for Time Series Classification
• Discrepancy-Aware Contrastive Adaptation in Medical Time Series Analysis
• PRISM: Lightweight Multivariate Time-Series Classification through Symmetric Multi-Resolution Convolutional Layers
• FeDaL: Federated Dataset Learning for Time Series Foundation Models
• CauKer: Classification Time Series Foundation Models Can Be Pretrained on Synthetic Data
• Manifold-regularised Large-Margin ℓ_p-SVDD for Multidimensional Time Series Anomaly Detection
• Hybrid Hypergraph Networks for Multimodal Sequence Data Classification
• A Contrastive Diffusion-based Network (CDNet) for Time Series Classification
• eMargin: Revisiting Contrastive Learning with Margin-Based Separation
• SDSC:A Structure-Aware Metric for Semantic Signal Representation Learning
• C3RL: Rethinking the Combination of Channel-independence and Channel-mixing from Representation Learning
• Mask6D: Masked Pose Priors For 6D Object Pose Estimation
• MoFE-Time: Mixture of Frequency Domain Experts for Time-Series Forecasting Models
• Multi-View Contrastive Learning for Robust Domain Adaptation in Medical Time Series Analysis
• ITFormer: Bridging Time Series and Natural Language for Multi-Modal QA with Large-Scale Multitask Dataset

Related

• Universal Time-Series Representation Learning: A Survey
• Unsupervised Representation Learning for Time Series: A Review
• T-Rep: Representation Learning for Time Series using Time-Embeddings
• Time-Series Representation Learning via Temporal and Contextual Contrasting
• Series2Vec: Similarity-based Self-supervised Representation Learning for Time Series Classification