2023/11/19 by Felix Pieper, Konstantin Ditschuneit, Pieper, Felix +7 · 3 citations
Computer Science · #Anomaly Detection Techniques and Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Time Series Analysis and Forecasting #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2311.11335
openalex publication_date 2023/11/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Self-supervised learning for time-series data holds potential similar to that recently unleashed in Natural Language Processing and Computer Vision. While most existing works in this area focus on contrastive learning, we propose a conceptually simple yet powerful non-contrastive approach, based on the data2vec self-distillation framework. The core of our method is a student-teacher scheme that predicts the latent representation of an input time series from masked views of the same time series. This strategy avoids strong modality-specific assumptions and biases typically introduced by the design of contrastive sample pairs. We demonstrate the competitiveness of our approach for classification and forecasting as downstream tasks, comparing with state-of-the-art self-supervised learning methods on the UCR and UEA archives as well as the ETT and Electricity datasets.