AdaRNN: Adaptive Learning and Forecasting of Time Series

2021/08/10 by Yuntao Du, Du, Yuntao, Jindong Wang +11 · 10 citations
Computer Science · Decision Sciences · #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Stock Market Forecasting Methods #Time Series Analysis and Forecasting

paper · pdf · doi:10.48550/arxiv.2108.04443

openalex publication_date 2021/08/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Time series has wide applications in the real world and is known to be difficult to forecast. Since its statistical properties change over time, its distribution also changes temporally, which will cause severe distribution shift problem to existing methods. However, it remains unexplored to model the time series in the distribution perspective. In this paper, we term this as Temporal Covariate Shift (TCS). This paper proposes Adaptive RNNs (AdaRNN) to tackle the TCS problem by building an adaptive model that generalizes well on the unseen test data. AdaRNN is sequentially composed of two novel algorithms. First, we propose Temporal Distribution Characterization to better characterize the distribution information in the TS. Second, we propose Temporal Distribution Matching to reduce the distribution mismatch in TS to learn the adaptive TS model. AdaRNN is a general framework with flexible distribution distances integrated. Experiments on human activity recognition, air quality prediction, and financial analysis show that AdaRNN outperforms the latest methods by a classification accuracy of 2.6% and significantly reduces the RMSE by 9.0%. We also show that the temporal distribution matching algorithm can be extended in Transformer structure to boost its performance.

Citations

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Domain Generalization via Invariant Feature Representation
• Deep CORAL: Correlation Alignment for Deep Domain Adaptation
• RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism
• Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting

Cited by

• MINED: Probing and Updating with Multimodal Time-Sensitive Knowledge for Large Multimodal Models
• Tackling Time-Series Forecasting Generalization via Mitigating Concept Drift
• KAIROS: Unified Training for Universal Non-Autoregressive Time Series Forecasting
• Inner-Instance Normalization for Time Series Forecasting
• From Patterns to Predictions: A Shapelet-Based Framework for Directional Forecasting in Noisy Financial Markets
• STA-GANN: A Valid and Generalizable Spatio-Temporal Kriging Approach
• Reinforcement Learning Driven Generalizable Feature Representation for Cross-User Activity Recognition
• DLGAN : Time Series Synthesis Based on Dual-Layer Generative Adversarial Networks
• ERIS: An Energy-Guided Feature Disentanglement Framework for Out-of-Distribution Time Series Classification
• EventTSF: Event-Aware Non-Stationary Time Series Forecasting

Related

• Robust Multivariate Time Series Forecasting against Intra- and Inter-Series Transitional Shift
• Explainable Parallel RCNN with Novel Feature Representation for Time Series Forecasting
• Dish-TS: A General Paradigm for Alleviating Distribution Shift in Time Series Forecasting
• Unlocking the Potential of Linear Networks for Irregular Multivariate Time Series Forecasting
• Persistence Initialization: A novel adaptation of the Transformer architecture for Time Series Forecasting