Meta-learning framework with applications to zero-shot time-series forecasting

2020/02/07 by Oreshkin, Boris N., Carpov, Dmitri, Chapados, Nicolas +1 · 6 citations
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.2002.02887

Abstract

Can meta-learning discover generic ways of processing time series (TS) from a diverse dataset so as to greatly improve generalization on new TS coming from different datasets? This work provides positive evidence to this using a broad meta-learning framework which we show subsumes many existing meta-learning algorithms. Our theoretical analysis suggests that residual connections act as a meta-learning adaptation mechanism, generating a subset of task-specific parameters based on a given TS input, thus gradually expanding the expressive power of the architecture on-the-fly. The same mechanism is shown via linearization analysis to have the interpretation of a sequential update of the final linear layer. Our empirical results on a wide range of data emphasize the importance of the identified meta-learning mechanisms for successful zero-shot univariate forecasting, suggesting that it is viable to train a neural network on a source TS dataset and deploy it on a different target TS dataset without retraining, resulting in performance that is at least as good as that of state-of-practice univariate forecasting models.

Cited by

• Quadratic Direct Forecast for Training Multi-Step Time-Series Forecast Models
• Chronos-2: From Univariate to Universal Forecasting
• A More Realistic Evaluation of Cross-Frequency Transfer Learning and Foundation Forecasting Models
• One-Embedding-Fits-All: Efficient Zero-Shot Time Series Forecasting by a Model Zoo
• TimeCopilot
• EnergyPatchTST: Multi-scale Time Series Transformers with Uncertainty Estimation for Energy Forecasting

Related

• Meta-Forecasting by combining Global Deep Representations with Local Adaptation
• Forecasting Early with Meta Learning
• Time series model selection with a meta-learning approach; evidence from a pool of forecasting algorithms
• Cross-Frequency Time Series Meta-Forecasting
• Dynamic Long-Term Time-Series Forecasting via Meta Transformer Networks