Meta-learning framework with applications to zero-shot time-series forecasting

2020/02/07 by Boris N. Oreshkin, Oreshkin, Boris N., Dmitri Carpov +5 · 11 citations
Computer Science · Environmental Science · Mathematics · #FOS: Computer and information sciences #Hydrological Forecasting Using AI #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Time Series Analysis and Forecasting #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2002.02887

openalex publication_date 2020/02/07 · arxiv created 2020/12/14 · arxiv updated 2020/12/16 · openalex created_date 2022/10/04 · openalex updated_date 2026/07/28

Abstract

Can meta-learning discover generic ways of processing time series (TS) from a diverse dataset so as to greatly improve generalization on new TS coming from different datasets? This work provides positive evidence to this using a broad meta-learning framework which we show subsumes many existing meta-learning algorithms. Our theoretical analysis suggests that residual connections act as a meta-learning adaptation mechanism, generating a subset of task-specific parameters based on a given TS input, thus gradually expanding the expressive power of the architecture on-the-fly. The same mechanism is shown via linearization analysis to have the interpretation of a sequential update of the final linear layer. Our empirical results on a wide range of data emphasize the importance of the identified meta-learning mechanisms for successful zero-shot univariate forecasting, suggesting that it is viable to train a neural network on a source TS dataset and deploy it on a different target TS dataset without retraining, resulting in performance that is at least as good as that of state-of-practice univariate forecasting models.

Cited by

• Quadratic Direct Forecast for Training Multi-Step Time-Series Forecast Models
• Chronos-2: From Univariate to Universal Forecasting
• A More Realistic Evaluation of Cross-Frequency Transfer Learning and Foundation Forecasting Models
• One-Embedding-Fits-All: Efficient Zero-Shot Time Series Forecasting by a Model Zoo
• TimeCopilot
• EnergyPatchTST: Multi-scale Time Series Transformers with Uncertainty Estimation for Energy Forecasting
• Rating Quality of Diverse Time Series Data by Meta-learning from LLM Judgment
• Generalisation Bounds of Zero-Shot Economic Forecasting using Time Series Foundation Models
• MetaSTNet: Multimodal Meta-learning for Cellular Traffic Conformal Prediction
• Context parroting: A simple but tough-to-beat baseline for foundation models in scientific machine learning
• DyGSSM: Multi-view Dynamic Graph Embeddings with State Space Model Gradient Update

Related

• Meta-Forecasting by combining Global Deep Representations with Local Adaptation
• Forecasting Early with Meta Learning
• Time series model selection with a meta-learning approach; evidence from a pool of forecasting algorithms
• Cross-Frequency Time Series Meta-Forecasting
• Dynamic Long-Term Time-Series Forecasting via Meta Transformer Networks