Only the Curve Shape Matters: Training Foundation Models for Zero-Shot Multivariate Time Series Forecasting through Next Curve Shape Prediction

2024/02/12 by Feng, Cheng, Huang, Long, Krompass, Denis · 3 citations
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2402.07570

Abstract

We present General Time Transformer (GTT), an encoder-only style foundation model for zero-shot multivariate time series forecasting. GTT is pretrained on a large dataset of 200M high-quality time series samples spanning diverse domains. In our proposed framework, the task of multivariate time series forecasting is formulated as a channel-wise next curve shape prediction problem, where each time series sample is represented as a sequence of non-overlapping curve shapes with a unified numerical magnitude. GTT is trained to predict the next curve shape based on a window of past curve shapes in a channel-wise manner. Experimental results demonstrate that GTT exhibits superior zero-shot multivariate forecasting capabilities on unseen time series datasets, even surpassing state-of-the-art supervised baselines. Additionally, we investigate the impact of varying GTT model parameters and training dataset scales, observing that the scaling law also holds in the context of zero-shot multivariate time series forecasting.

Cited by

• Foundation Model Forecasts: Form and Function
• VisionTS++: Cross-Modal Time Series Foundation Model with Continual Pre-trained Vision Backbones
• Foundation models for time series forecasting: Application in conformal prediction

Related

• TimePFN: Effective Multivariate Time Series Forecasting with Synthetic Data
• TimeFound: A Foundation Model for Time Series Forecasting
• Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting
• A Mamba Foundation Model for Time Series Forecasting
• Unified Training of Universal Time Series Forecasting Transformers