MOMENT: A Family of Open Time-series Foundation Models

2024/02/06 by Mononito Goswami, Goswami, Mononito, Konrad Szafer +10 · 1 voice · 104 citations
Computer Science · Decision Sciences · #Simulation Techniques and Applications #Time Series Analysis and Forecasting #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.2402.03885

openalex publication_date 2024/02/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We introduce MOMENT, a family of open-source foundation models for general-purpose time series analysis. Pre-training large models on time series data is challenging due to (1) the absence of a large and cohesive public time series repository, and (2) diverse time series characteristics which make multi-dataset training onerous. Additionally, (3) experimental benchmarks to evaluate these models, especially in scenarios with limited resources, time, and supervision, are still in their nascent stages. To address these challenges, we compile a large and diverse collection of public time series, called the Time series Pile, and systematically tackle time series-specific challenges to unlock large-scale multi-dataset pre-training. Finally, we build on recent work to design a benchmark to evaluate time series foundation models on diverse tasks and datasets in limited supervision settings. Experiments on this benchmark demonstrate the effectiveness of our pre-trained models with minimal data and task-specific fine-tuning. Finally, we present several interesting empirical observations about large pre-trained time series models. Pre-trained models (AutonLab/MOMENT-1-large) and Time Series Pile (AutonLab/Timeseries-PILE) are available on Huggingface.

Cited by

• NeuCoReClass AD: Redefining self-supervised time series anomaly detection
• Post-Training in Time Series Foundation Models: A Unifying Framework
• Trend strength predicts when generative foundation models win: a power-controlled benchmark, a mechanism, and an actionable selection rule
• Learning Spatio-Temporal Foundation Models from Pure Synthetic Data
• Are Time-Indexed Foundation Models the Future of Time Series Imputation?
• Accelerating Time Series Foundation Models with Speculative Decoding
• TS-Arena Technical Report -- A Pre-registered Live Forecasting Platform
• The Forecast Critic: Leveraging Large Language Models for Poor Forecast Identification
• Optimizing the Training Diet: Data Mixture Search for Robust Time Series Forecasting
• Stanford Sleep Bench: Evaluating Polysomnography Pre-training Methods for Sleep Foundation Models
• Weighted Contrastive Learning for Anomaly-Aware Time-Series Forecasting
• TinyMyo: a Tiny Foundation Model for Flexible EMG Signal Processing at the Edge
• TS-HINT: Enhancing Semiconductor Time Series Regression Using Attention Hints From Large Language Model Reasoning
• A Comparative Study on How Data Normalization Affects Zero-Shot Generalization in Time Series Foundation Models
• CLEF: Clinically-Guided Contrastive Learning for Electrocardiogram Foundation Models
• FMTK: A Modular Toolkit for Composable Time Series Foundation Model Pipelines
• FiCoTS: Fine-to-Coarse LLM-Enhanced Hierarchical Cross-Modality Interaction for Time Series Forecasting
• Data-Efficient Motor Condition Monitoring with Time Series Foundation Models
• An Electrocardiogram Multi-task Benchmark with Comprehensive Evaluations and Insightful Findings
• EnECG: Efficient Ensemble Learning for Electrocardiogram Multi-task Foundation Model
• Time Series Forecasting via Direct Per-Step Probability Distribution Modeling
• On the Internal Semantics of Time-Series Foundation Models
• HIT-ROCKET: Hadamard-vector Inner-product Transformer for ROCKET
• STAMP: Spatial-Temporal Adapter with Multi-Head Pooling
• Moirai 2.0: When Less Is More for Time Series Forecasting
• PE-TSFM: Self-Supervised Time-Series Learning for Generalizable Power Converter Health Monitoring under Unseen Conditions
• Lightweight Time Series Data Valuation on Time Series Foundation Models via In-Context Finetuning
• A Theoretical Analysis of Detecting Large Model-Generated Time Series
• The Few Govern the Many:Unveiling Few-Layer Dominance for Time Series Models
• Predicting the Future by Retrieving the Past
• Adaptation and Fine-tuning with TabPFN for Travelling Salesman Problem
• Evaluating Spatio-Temporal Forecasting Trade-offs Between Graph Neural Networks and Foundation Models
• Frequency Matters: When Time Series Foundation Models Fail Under Spectral Shift
• Forecast2Anomaly (F2A): Adapting Multivariate Time Series Foundation Models for Anomaly Prediction
• Reducing Robotic Upper-Limb Assessment Time While Maintaining Precision: A Time Series Foundation Model Approach
• Leveraging Generic Time Series Foundation Models for EEG Classification
• Pre-trained Forecasting Models: Strong Zero-Shot Feature Extractors for Time Series Classification
• SignalMC-MED: A Multimodal Benchmark for Evaluating Biosignal Foundation Models on Single-Lead ECG and PPG
• TS-Reasoner: Aligning Time Series Foundation Models with LLM Reasoning
• SwiftTS: A Swift Selection Framework for Time Series Pre-trained Models via Multi-task Meta-Learning
• DBLoss: Decomposition-based Loss Function for Time Series Forecasting
• Learning Reconfigurable Representations for Multimodal Federated Learning with Missing Data
• NetBurst: Event-Centric Forecasting of Bursty, Intermittent Time Series
• There is No "apple" in Timeseries: Rethinking TSFM through the Lens of Invariance
• SEMPO: Lightweight Foundation Models for Time Series Forecasting
• Foundation Model Forecasts: Form and Function
• Generalist vs Specialist Time Series Foundation Models: Investigating Potential Emergent Behaviors in Assessing Human Health Using PPG Signals
• Rethinking Evaluation in the Era of Time Series Foundation Models: (Un)known Information Leakage Challenges
• STAR: Boosting Time Series Foundation Models for Anomaly Detection through State-aware Adapter
• CrossAD: Time Series Anomaly Detection with Cross-scale Associations and Cross-window Modeling
• DiffStyleTS: Diffusion Model for Style Transfer in Time Series
• Vision-LLMs for Spatiotemporal Traffic Forecasting
• Vision4PPG: Emergent PPG Analysis Capability of Vision Foundation Models for Vital Signs like Blood Pressure
• MLLM4TS: Leveraging Vision and Multimodal Language Models for General Time-Series Analysis
• Rapid prediction of cerebral edema on CT scan after traumatic brain injury
• Pool Me Wisely: On the Effect of Pooling in Transformer-Based Models
• TelecomTS: A Multi-Modal Observability Dataset for Time Series and Language Analysis
• CarbonX: An Open-Source Tool for Computational Decarbonization Using Time Series Foundation Models
• Eliciting Chain-of-Thought Reasoning for Time Series Analysis using Reinforcement Learning
• Black-Box Time-Series Domain Adaptation via Cross-Prompt Foundation Models
• TsLLM: Augmenting LLMs for General Time Series Understanding and Prediction
• Kairos: Towards Adaptive and Generalizable Time Series Foundation Models
• Joint Embeddings Go Temporal
• AXIS: Explainable Time Series Anomaly Detection with Large Language Models
• FM-FoG: A Real-Time Foundation Model-based Wearable System for Freezing-of-Gait Mitigation
• Estimating Time Series Foundation Model Transferability via In-Context Learning
• Mind the Missing: Variable-Aware Representation Learning for Irregular EHR Time Series using Large Language Models
• Beyond the Clinic: A Large-Scale Evaluation of Augmenting EHR with Wearable Data for Diverse Health Prediction
• Towards Foundation Models for Zero-Shot Time Series Anomaly Detection: Leveraging Synthetic Data and Relative Context Discrepancy
• VFEM: Visual Feature Empowered Multivariate Time Series Forecasting with Cross-Modal Fusion
• DiffNator: Generating Structured Explanations of Time-Series Differences
• EEG-FM-Bench: A Comprehensive Benchmark for the Systematic Evaluation of EEG Foundation Models
• TimeMosaic: Temporal Heterogeneity Guided Time Series Forecasting via Adaptive Granularity Patch and Segment-wise Decoding
• MOMEMTO: Patch-based Memory Gate Model in Time Series Foundation Model
• TSGym: Design Choices for Deep Multivariate Time-Series Forecasting
• DeCoP: Enhancing Self-Supervised Time Series Representation with Dependency Controlled Pre-training
• Super-Linear: A Lightweight Pretrained Mixture of Linear Experts for Time Series Forecasting
• TFMAdapter: Lightweight Instance-Level Adaptation of Foundation Models for Forecasting with Covariates
• Leveraging Intermediate Representations of Time Series Foundation Models for Anomaly Detection
• A Time-Series Foundation Model by Universal Delay Embedding
• Large language models surpass domain-specific architectures for antepartum electronic fetal monitoring analysis
• WindFM: An Open-Source Foundation Model for Zero-Shot Wind Power Forecasting
• ARIES: Relation Assessment and Model Recommendation for Deep Time Series Forecasting
• One-Embedding-Fits-All: Efficient Zero-Shot Time Series Forecasting by a Model Zoo
• A User-centric Kubernetes-based Architecture for Green Cloud Computing
• TimeCopilot
• CALM: A Framework for Continuous, Adaptive, and LLM-Mediated Anomaly Detection in Time-Series Streams
• On Identifying Why and When Foundation Models Perform Well on Time-Series Forecasting Using Automated Explanations and Rating
• Robustness is Important: Limitations of LLMs for Data Fitting
• Enhancing Transformer-Based Foundation Models for Time Series Forecasting via Bagging, Boosting and Statistical Ensembles
• Enhancing Cryptocurrency Sentiment Analysis with Multimodal Features
• An Experimental Reservoir-Augmented Foundation Model: 6G O-RAN Case Study
• Retrieval-Augmented Water Level Forecasting for Everglades
• VisionTS++: Cross-Modal Time Series Foundation Model with Continual Pre-trained Vision Backbones
• FeDaL: Federated Dataset Learning for Time Series Foundation Models
• ZARA: Zero-shot Motion Time-Series Analysis via Knowledge and Retrieval Driven LLM Agents
• CauKer: Classification Time Series Foundation Models Can Be Pretrained on Synthetic Data
• Is It Really You? Exploring Biometric Verification Scenarios in Photorealistic Talking-Head Avatar Videos
• Dynamics is what you need for time-series forecasting!
• MoTM: Towards a Foundation Model for Time Series Imputation based on Continuous Modeling
• U-Cast: Learning Hierarchical Structures for High-Dimensional Time Series Forecasting
• Time-RA: Towards Time Series Reasoning for Anomaly with LLM Feedback
• ParallelTime: Dynamically Weighting the Balance of Short- and Long-Term Temporal Dependencies
• Hallucination Detection and Mitigation with Diffusion in Multi-Variate Time-Series Foundation Models

Discussions

• Moment: A Family of Open Time-Series Foundation Models [hn, 55 points, 5 comments]

Related

• Empowering Time Series Analysis with Foundation Models: A Comprehensive Survey
• Foundation Models for Time Series: A Survey
• Empowering Time Series Analysis with Synthetic Data: A Survey and Outlook in the Era of Foundation Models
• Towards Foundation Time Series Model: To Synthesize Or Not To Synthesize?
• How Foundational are Foundation Models for Time Series Forecasting?