Chronos: Learning the Language of Time Series

2024/03/12 by Abdul Fatir Ansari, Ansari, Abdul Fatir, Lorenzo Stella +36 · 4 voices · 156 citations
Computer Science · Physics and Astronomy · #Time Series Analysis and Forecasting #Advanced Text Analysis Techniques #Historical Astronomy and Related Studies

paper · pdf · doi:10.48550/arxiv.2403.07815

Abstract

We introduce Chronos, a simple yet effective framework for pretrained probabilistic time series models. Chronos tokenizes time series values using scaling and quantization into a fixed vocabulary and trains existing transformer-based language model architectures on these tokenized time series via the cross-entropy loss. We pretrained Chronos models based on the T5 family (ranging from 20M to 710M parameters) on a large collection of publicly available datasets, complemented by a synthetic dataset that we generated via Gaussian processes to improve generalization. In a comprehensive benchmark consisting of 42 datasets, and comprising both classical local models and deep learning methods, we show that Chronos models: (a) significantly outperform other methods on datasets that were part of the training corpus; and (b) have comparable and occasionally superior zero-shot performance on new datasets, relative to methods that were trained specifically on them. Our results demonstrate that Chronos models can leverage time series data from diverse domains to improve zero-shot accuracy on unseen forecasting tasks, positioning pretrained models as a viable tool to greatly simplify forecasting pipelines.

Cited by

• From Vector Autoregressions to AI-based Time Series Forecasting: A Review
• Expert-Guided Forecast Editing for Time-Series Foundation Models
• Zero-Shot Heart Rate Variability Forecasting from Consumer Wearables Using Time Series Foundation Models
• A Three-Phase Foundation Model for Tax-Aware Personalized Portfolio Management
• Post-Training in Time Series Foundation Models: A Unifying Framework
• SKETCH: Semantic Key-Point Conditioning for Long-Horizon Vessel Trajectory Prediction
• Residual-Guided Multi-Resolution Refinement of Foundation Models: A Case Study in Drought Forecasting
• VLT: A Vision-Language-Time Series Multimodal Foundation Model for Industrial Intelligence
• A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems
• VAN-AD: Visual Masked Autoencoder with Normalizing Flow For Time Series Anomaly Detection
• Air Quality Arena: A Large-Scale Multi-Region Ground Monitoring Dataset and Benchmark for Air Quality Forecasting with Time-Series Foundation Models
• Trend strength predicts when generative foundation models win: a power-controlled benchmark, a mechanism, and an actionable selection rule
• Learning Spatio-Temporal Foundation Models from Pure Synthetic Data
• FinBench: Time-Gated Calibration and Uncertainty Benchmarking for Agentic Financial Forecasting
• Transformers for dynamical systems learn transfer operators in-context
• Position: A Dynamical Systems Perspective is Needed to Advance Time Series Modeling
• What Neuroscience Can Teach AI About Learning in Continuously Changing Environments
• Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictions
• Panda: A pretrained forecast model for chaotic dynamics
• True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statistics
• Are Time-Indexed Foundation Models the Future of Time Series Imputation?
• Accelerating Time Series Foundation Models with Speculative Decoding
• Time Series Foundation Models for Process Model Forecasting
• Autoregressive Flow Matching for Motion Prediction
• Chain-of-thought Reviewing and Correction for Time Series Question Answering
• QuantFlow: A Federated Mamba-Based Post-Transformer Foundation Model for Time-Series Forecasting
• LiFT-MPC: Language-in-the-Loop Feedback Tuning of Cost Previews for MPC
• Foundation Models and Fine-Tuning: Toward a New Generation of Models for Time Series Forecasting
• LLM as Forecasting Planner: Training-Free Text Conditioning for Time-Series Foundation Models
• SeT-Diff: Towards Semantic Foundation Models for HPC Telemetry and Time-Series
• RefineBridge: Generative Bridge Models Improve Financial Forecasting by Foundation Models
• The Seismic Wavefield Common Task Framework
• FLAME: Flow Enhanced Legendre Memory Models for General Time Series Forecasting
• Mitigating Exposure Bias in Risk-Aware Time Series Forecasting with Soft Tokens
• BERTO: an Adaptive BERT-based Network Time Series Predictor with Operator Preferences in Natural Language
• A Comparative Study on How Data Normalization Affects Zero-Shot Generalization in Time Series Foundation Models
• FMTK: A Modular Toolkit for Composable Time Series Foundation Model Pipelines
• Auditable Context-Aware HFMD Forecasting with Structured LLM Agents
• Modèles de Fondation et Ajustement : Vers une Nouvelle Génération de Modèles pour la Prévision des Séries Temporelles
• Learning Time-Scale Invariant Population-Level Neural Representations
• Zero-Shot Transfer Capabilities of the Sundial Foundation Model for Leaf Area Index Forecasting
• Temporal-Visual Semantic Alignment: A Unified Architecture for Transferring Spatial Priors from Vision Models to Zero-Shot Temporal Tasks
• Cisco Time Series Model Technical Report
• TiCT: A Synthetically Pre-Trained Foundation Model for Time Series Classification
• Structured Noise Modeling for Enhanced Time-Series Forecasting
• Scaling Conditional Autoencoders for Portfolio Optimization via Uncertainty-Aware Factor Selection
• Multi-layer Stack Ensembles for Time Series Forecasting
• On the Internal Semantics of Time-Series Foundation Models
• Genomic Next-Token Predictors are In-Context Learners
• STAMP: Spatial-Temporal Adapter with Multi-Head Pooling
• Leveraging Exogenous Signals for Hydrology Time Series Forecasting
• Moirai 2.0: When Less Is More for Time Series Forecasting
• Review of Passenger Flow Modelling Approaches Based on a Bibliometric Analysis
• AlphaCast: A Human Wisdom-LLM Intelligence Co-Reasoning Framework for Interactive Time Series Forecasting
• OmniFuser: Adaptive Multimodal Fusion for Service-Oriented Predictive Maintenance
• The Few Govern the Many:Unveiling Few-Layer Dominance for Time Series Models
• Synapse: Adaptive Arbitration of Complementary Expertise in Time Series Foundational Models
• Frequency Matters: When Time Series Foundation Models Fail Under Spectral Shift
• Small Vocabularies, Big Gains: Pretraining and Tokenization in Time Series Models
• Agentic World Modeling for 6G: Near-Real-Time Generative State-Space Reasoning
• Reducing Robotic Upper-Limb Assessment Time While Maintaining Precision: A Time Series Foundation Model Approach
• Leveraging Generic Time Series Foundation Models for EEG Classification
• Pre-trained Forecasting Models: Strong Zero-Shot Feature Extractors for Time Series Classification
• TempoPFN: Synthetic Pre-training of Linear RNNs for Zero-shot Time Series Forecasting
• Safety-Gated Autoscaling: A Multi-Layered Defense Architecture for Kubernetes Vertical Resource Optimization
• TS-Reasoner: Aligning Time Series Foundation Models with LLM Reasoning
• Selective Learning for Deep Time Series Forecasting
• Time-Series Foundation Models for ISP Traffic Forecasting
• SwiftTS: A Swift Selection Framework for Time Series Pre-trained Models via Multi-task Meta-Learning
• Learning Reconfigurable Representations for Multimodal Federated Learning with Missing Data
• NetBurst: Event-Centric Forecasting of Bursty, Intermittent Time Series
• Chronos-2: From Univariate to Universal Forecasting
• There is No "apple" in Timeseries: Rethinking TSFM through the Lens of Invariance
• xTime: Extreme Event Prediction with Hierarchical Knowledge Distillation and Expert Fusion
• SEMPO: Lightweight Foundation Models for Time Series Forecasting
• Online Kernel Dynamic Mode Decomposition for Streaming Time Series Forecasting with Adaptive Windowing
• Benchmarking Probabilistic Time Series Forecasting Models on Neural Activity
• Xihe: Scalable Zero-Shot Time Series Learner Via Hierarchical Interleaved Block Attention
• Beyond Accuracy: Are Time Series Foundation Models Well-Calibrated?
• Generalist vs Specialist Time Series Foundation Models: Investigating Potential Emergent Behaviors in Assessing Human Health Using PPG Signals
• Rethinking Evaluation in the Era of Time Series Foundation Models: (Un)known Information Leakage Challenges
• Toward Reasoning-Centric Time-Series Analysis
• CoRA: Covariate-Aware Adaptation of Time Series Foundation Models
• Leveraging Language Semantics for Collaborative Filtering with TextGCN and TextGCN-MLP: Zero-Shot vs In-Domain Performance
• Vision-LLMs for Spatiotemporal Traffic Forecasting
• A Comprehensive Forecasting-Based Framework for Time Series Anomaly Detection: Benchmarking on the Numenta Anomaly Benchmark (NAB)
• KAIROS: Unified Training for Universal Non-Autoregressive Time Series Forecasting
• MLLM4TS: Leveraging Vision and Multimodal Language Models for General Time-Series Analysis
• Test-Time Efficient Pretrained Model Portfolios for Time Series Forecasting
• TelecomTS: A Multi-Modal Observability Dataset for Time Series and Language Analysis
• TimeSeriesScientist: A General-Purpose AI Agent for Time Series Analysis
• Benchmarking M-LTSF: Frequency and Noise-Based Evaluation of Multivariate Long Time Series Forecasting Models
• Understanding Transformers for Time Series: Rank Structure, Flow-of-ranks, and Compressibility
• CarbonX: An Open-Source Tool for Computational Decarbonization Using Time Series Foundation Models
• Accuracy Law for the Future of Deep Time Series Forecasting
• TsLLM: Augmenting LLMs for General Time Series Understanding and Prediction
• Foundation vs. Specialized Models: Evaluating Catastrophic Forgetting in Continual Time Series Forecasting
• fev-bench: A Realistic Benchmark for Time Series Forecasting
• EntroPE: Entropy-Guided Dynamic Patch Encoder for Time Series Forecasting
• Kairos: Towards Adaptive and Generalizable Time Series Foundation Models
• Uncovering Zero-Shot Generalization Gaps in Time-Series Foundation Models Using Real-World Videos
• Fidel-TS: A High-Fidelity Benchmark for Multimodal Time Series Forecasting
• AXIS: Explainable Time Series Anomaly Detection with Large Language Models
• TimeExpert: Boosting Long Time Series Forecasting with Temporal Mix of Experts
• Aurora: Towards Universal Generative Multimodal Time Series Forecasting
• SciTS: Scientific Time Series Understanding and Generation with LLMs
• ChaosNexus: A Foundation Model for Universal Chaotic System Forecasting with Multi-scale Representations
• Towards Foundation Models for Zero-Shot Time Series Anomaly Detection: Leveraging Synthetic Data and Relative Context Discrepancy
• CaTS-Bench: Can Language Models Describe Numeric Time Series?
• Look Before you Leap: Estimating LLM Benchmark Scores from Descriptions
• Why Attention Fails: The Degeneration of Transformers into MLPs in Time Series Forecasting
• VFEM: Visual Feature Empowered Multivariate Time Series Forecasting with Cross-Modal Fusion
• DiffNator: Generating Structured Explanations of Time-Series Differences
• Multimodal Language Models with Modality-Specific Experts for Financial Forecasting from Interleaved Sequences of Text and Time Series
• A novel hybrid approach to contraceptive demand forecasting: integrating point predictions with probabilistic distributions
• SleepLM: Natural-Language Intelligence for Human Sleep
• OSF: On Pre-training and Scaling of Sleep Foundation Models
• GCT: A Granger-Causal Transformer for Multivariate Traffic Analysis in Smart Villages
• MOMEMTO: Patch-based Memory Gate Model in Time Series Foundation Model
• A Unified AI Approach for Continuous Monitoring of Human Health and Diseases from Intensive Care Unit to Home with Physiological Foundation Models (UNIPHY+)
• Super-Linear: A Lightweight Pretrained Mixture of Linear Experts for Time Series Forecasting
• TFMAdapter: Lightweight Instance-Level Adaptation of Foundation Models for Forecasting with Covariates
• PiERN: Token-Level Routing for Integrating High-Precision Computation and Reasoning
• GraphMend: Code Transformations for Fixing Graph Breaks in PyTorch 2
• A Time-Series Foundation Model by Universal Delay Embedding
• Customizing the Inductive Biases of Softmax Attention using Structured Matrices
• WindFM: An Open-Source Foundation Model for Zero-Shot Wind Power Forecasting
• Methodological Insights into Structural Causal Modelling and Uncertainty-Aware Forecasting for Economic Indicators
• ARIES: Relation Assessment and Model Recommendation for Deep Time Series Forecasting
• Select, then Balance: Exploring Exogenous Variable Modeling of Spatio-Temporal Forecasting
• Scaling Law for Large-Scale Pre-Training Using Chaotic Time Series and Predictability in Financial Time Series
• ChronoGraph: A Real-World Graph-Based Multivariate Time Series Dataset
• One-Embedding-Fits-All: Efficient Zero-Shot Time Series Forecasting by a Model Zoo
• Probabilistic Pretraining for Neural Regression
• RDIT: Residual-based Diffusion Implicit Models for Probabilistic Time Series Forecasting
• A User-centric Kubernetes-based Architecture for Green Cloud Computing
• Text Reinforcement for Multimodal Time Series Forecasting
• Cross-device Zero-shot Label Transfer via Alignment of Time Series Foundation Model Embeddings
• CALM: A Framework for Continuous, Adaptive, and LLM-Mediated Anomaly Detection in Time-Series Streams
• Adapting to Change: A Comparison of Continual and Transfer Learning for Modeling Building Thermal Dynamics under Concept Drifts
• On Identifying Why and When Foundation Models Perform Well on Time-Series Forecasting Using Automated Explanations and Rating
• Enhancing Transformer-Based Foundation Models for Time Series Forecasting via Bagging, Boosting and Statistical Ensembles
• CC-Time: Cross-Model and Cross-Modality Time Series Forecasting
• Measuring Time Series Forecast Stability for Demand Planning
• Comparative Analysis of Time Series Foundation Models for Demographic Forecasting: Enhancing Predictive Accuracy in US Population Dynamics
• FlowState: Sampling Rate Invariant Time Series Forecasting
• Retrieval-Augmented Water Level Forecasting for Everglades
• PriceFM: Foundation Model for Probabilistic Electricity Price Forecasting
• VisionTS++: Cross-Modal Time Series Foundation Model with Continual Pre-trained Vision Backbones
• STARE: Predicting Decision Making Based on Spatio-Temporal Eye Movements
• FeDaL: Federated Dataset Learning for Time Series Foundation Models
• ZARA: Zero-shot Motion Time-Series Analysis via Knowledge and Retrieval Driven LLM Agents
• HGTS-Former: Hierarchical HyperGraph Transformer for Multivariate Time Series Analysis
• Kronos: A Foundation Model for the Language of Financial Markets
• KFS: KAN based adaptive Frequency Selection learning architecture for long term time series forecasting
• Foundation Models for Clean Energy Forecasting: A Comprehensive Review

Discussions

• Chronos: Learning the Language of Time Series [hn, 207 points, 59 comments]
• Chronos: New ML Framework for Pretrained Probabilistic Time Series Models [hn, 9 points, 0 comments]
• Chronos: Learning the Language of Time Series [hn, 6 points, 1 comments]
• Unhealthy saturday night reading. LLM (llama) trained to do time series forecasting. Beats (auto)arima etc quite a bit. Exchange rate, macro, retail sales, bike rides, weather etc datasets. https://a [bsky, 0 points, 1 comments]

Related

• Chronos-2: From Univariate to Universal Forecasting
• The Relevance of AWS Chronos: An Evaluation of Standard Methods for Time Series Forecasting with Limited Tuning
• TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting
• ChronoSteer: Bridging Large Language Model and Time Series Foundation Model via Synthetic Data
• Time-LLM: Time Series Forecasting by Reprogramming Large Language Models