Meta-Learning Dynamics Forecasting Using Task Inference

2021/02/20 by Rui Wang, Wang, Rui, Robin Walters +3 · 5 citations
Earth and Planetary Sciences · Engineering · Environmental Science · #FOS: Computer and information sciences #Hydrological Forecasting Using AI #Machine Learning (cs.LG) #Meteorological Phenomena and Simulations #Reservoir Engineering and Simulation Methods

paper · pdf · doi:10.48550/arxiv.2102.10271

openalex publication_date 2021/02/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Current deep learning models for dynamics forecasting struggle with generalization. They can only forecast in a specific domain and fail when applied to systems with different parameters, external forces, or boundary conditions. We propose a model-based meta-learning method called DyAd which can generalize across heterogeneous domains by partitioning them into different tasks. DyAd has two parts: an encoder which infers the time-invariant hidden features of the task with weak supervision, and a forecaster which learns the shared dynamics of the entire domain. The encoder adapts and controls the forecaster during inference using adaptive instance normalization and adaptive padding. Theoretically, we prove that the generalization error of such procedure is related to the task relatedness in the source domain, as well as the domain differences between source and target. Experimentally, we demonstrate that our model outperforms state-of-the-art approaches on both turbulent flow and real-world ocean data forecasting tasks.

Citations

• The Benefit of Multitask Representation Learning
• Prototypical Networks for Few-shot Learning
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations
• Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
• Meta-Reinforcement Learning of Structured Exploration Strategies
• Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
• Learning Dynamical Systems from Partial Observations
• Hierarchically Structured Meta-learning
• N-BEATS: Neural basis expansion analysis for interpretable time series forecasting
• Incorporating Symmetry into Deep Dynamics Models for Improved Generalization
• Physics-aware Spatiotemporal Modules with Auxiliary Tasks for Meta-Learning

Cited by

• Video Prediction of Dynamic Physical Simulations With Pixel-Space Spatiotemporal Transformers
• Environment Inference for Learning Generalizable Dynamical System
• Towards Generalizable PDE Dynamics Forecasting via Physics-Guided Invariant Learning
• DroneFL: Federated Learning for Multi-UAV Visual Target Tracking
• Generalizing to New Dynamical Systems via Frequency Domain Adaptation

Related

• Tailored Forecasting from Short Time Series via Meta-learning
• A Meta-learning Approach to Reservoir Computing: Time Series Prediction with Limited Data
• Forecasting Early with Meta Learning
• Dynamics is what you need for time-series forecasting!
• Meta-learning framework with applications to zero-shot time-series forecasting