Are Transformers Effective for Time Series Forecasting?
2022/05/26 by Ailing Zeng, Zeng, Ailing, Muxi Chen +5 · 1 voice · 218 citations
Computer Science · #Advanced Text Analysis Techniques #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Time Series Analysis and Forecasting #cs.AI #cs.LG
paper · pdf · doi:10.48550/arxiv.2205.13504
openalex publication_date 2022/05/26 · arxiv published 2022/05/26 · arxiv updated 2022/08/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Abstract
Recently, there has been a surge of Transformer-based solutions for the long-term time series forecasting (LTSF) task. Despite the growing performance over the past few years, we question the validity of this line of research in this work. Specifically, Transformers is arguably the most successful solution to extract the semantic correlations among the elements in a long sequence. However, in time series modeling, we are to extract the temporal relations in an ordered set of continuous points. While employing positional encoding and using tokens to embed sub-series in Transformers facilitate preserving some ordering information, the nature of the permutation-invariant self-attention mechanism inevitably results in temporal information loss. To validate our claim, we introduce a set of embarrassingly simple one-layer linear models named LTSF-Linear for comparison. Experimental results on nine real-life datasets show that LTSF-Linear surprisingly outperforms existing sophisticated Transformer-based LTSF models in all cases, and often by a large margin. Moreover, we conduct comprehensive empirical studies to explore the impacts of various design elements of LTSF models on their temporal relation extraction capability. We hope this surprising finding opens up new research directions for the LTSF task. We also advocate revisiting the validity of Transformer-based solutions for other time series analysis tasks (e.g., anomaly detection) in the future. Code is available at: \urlhttps://github.com/cure-lab/LTSF-Linear.
Cited by
- What Matters in Deep Learning for Time Series Forecasting?
- TimePerceiver: An Encoder-Decoder Framework for Generalized Time-Series Forecasting
- Multivariate Time Series Forecasting with Adaptive Non-Local Observables
- Optimizing Transformer Neural Network for Real-Time Outlier Detection on FPGAs
- Foundation Models for EEG Are Blind to Long-Range Temporal Correlations: A Spectral-Temporal Dissociation Behind Their Cross-Population Fragility
- Atmospheric Diffusion-Guided Spatio-Temporal Transformer for Nuclear Radiation Forecasting
- Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting
- Dual-Prototype Disentanglement: A Context-Aware Enhancement Framework for Time Series Forecasting
- RefineBridge: Generative Bridge Models Improve Financial Forecasting by Foundation Models
- HGAN-SDEs: Learning Neural Stochastic Differential Equations with Hermite-Guided Adversarial Training
- DecoKAN: Interpretable Decomposition for Forecasting Cryptocurrency Market Dynamics
- LoFT-LLM: Low-Frequency Time-Series Forecasting with Large Language Models
- A lightweight Spatial-Temporal Graph Neural Network for Long-term Time Series Forecasting
- Reducing Pilots in Channel Estimation with Predictive Foundation Models
- Physics-driven human-like working memory outperforms digital networks in dynamic vision
- FLAME: Flow Enhanced Legendre Memory Models for General Time Series Forecasting
- Multivariate Time Series Forecasting with Hybrid Euclidean-SPD Manifold Graph Neural Networks
- Neural CDEs as Correctors for Learned Time Series Models
- Mitigating Exposure Bias in Risk-Aware Time Series Forecasting with Soft Tokens
- DB2-TransF: All You Need Is Learnable Daubechies Wavelets for Time Series Forecasting
- Adaptive Information Routing for Multimodal Time Series Forecasting
- Bridging the Clinical Expertise Gap: Development of a Web-Based Platform for Accessible Time Series Forecasting and Analysis
- FRWKV:Frequency-Domain Linear Attention for Long-Term Time Series Forecasting
- Adaptive Normalization Mamba with Multi Scale Trend Decomposition and Patch MoE Encoding
- Generative AI for Self-Adaptive Systems: State of the Art and Research Roadmap
- A Multi-View Multi-Timescale Hypergraph-Empowered Spatiotemporal Framework for EV Charging Forecasting
- D-CTNet: A Dual-Branch Channel-Temporal Forecasting Network with Frequency-Domain Correction
- DPWMixer: Dual-Path Wavelet Mixer for Long-Term Time Series Forecasting
- FiCoTS: Fine-to-Coarse LLM-Enhanced Hierarchical Cross-Modality Interaction for Time Series Forecasting
- Ada-MoGE: Adaptive Mixture of Gaussian Expert Model for Time Series Forecasting
- An Electrocardiogram Multi-task Benchmark with Comprehensive Evaluations and Insightful Findings
- TARFVAE: Efficient One-Step Generative Time Series Forecasting via TARFLOW based VAE
- EnECG: Efficient Ensemble Learning for Electrocardiogram Multi-task Foundation Model
- Modèles de Fondation et Ajustement : Vers une Nouvelle Génération de Modèles pour la Prévision des Séries Temporelles
- TS2Vec-Ensemble: An Enhanced Self-Supervised Framework for Time Series Forecasting
- RED-F: Reconstruction-Elimination based Dual-stream Contrastive Forecasting for Multivariate Time Series Anomaly Prediction
- MSTN: Fast and Efficient Multivariate Time Series Prediction Model
- The Driver-Blindness Phenomenon: Why Deep Sequence Models Default to Autocorrelation in Blood Glucose Forecasting
- Temporal-Visual Semantic Alignment: A Unified Architecture for Transferring Spatial Priors from Vision Models to Zero-Shot Temporal Tasks
- TimePre: Bridging Accuracy, Efficiency, and Stability in Probabilistic Time-Series Forecasting
- PeriodNet: Boosting the Potential of Attention Mechanism for Time Series Forecasting
- Partial multivariate transformer as a tool for cryptocurrencies time series prediction
- Multi-layer Stack Ensembles for Time Series Forecasting
- Adapformer: Adaptive Channel Management for Multivariate Time Series Forecasting
- Naga: Vedic Encoding for Deep State Space Models
- TabulaTime: Novel multimodal deep learning for Acute Coronary Syndrome prediction through environmental and clinical data integration
- VitalBench: A Rigorous Multi-Center Benchmark for Long-Term Vital Sign Prediction in Intraoperative Care
- Moirai 2.0: When Less Is More for Time Series Forecasting
- DenoGrad: A Gradient-Based Framework for Data Refinement in Tabular and Time-Series Learning
- AlphaCast: A Human Wisdom-LLM Intelligence Co-Reasoning Framework for Interactive Time Series Forecasting
- Trend-Aware Multi-Task Learning for Short-Term Energy Forecasting
- Filtering Jump Markov Systems with Partially Known Dynamics: A Model-Based Deep Learning Approach
- Lightweight Time Series Data Valuation on Time Series Foundation Models via In-Context Finetuning
- COGNOS: Universal Enhancement for Time Series Anomaly Detection via Constrained Gaussian-Noise Optimization and Smoothing
- The Few Govern the Many:Unveiling Few-Layer Dominance for Time Series Models
- AWEMixer: Adaptive Wavelet-Enhanced Mixer Network for Long-Term Time Series Forecasting
- Synapse: Adaptive Arbitration of Complementary Expertise in Time Series Foundational Models
- Reasoning on Time-Series for Financial Technical Analysis
- Multi-period Learning for Financial Time Series Forecasting
- Electric Vehicle Charging Load Modeling: A Survey, Trends, Challenges and Opportunities
- TempoPFN: Synthetic Pre-training of Linear RNNs for Zero-shot Time Series Forecasting
- Do We Really Need Adaptive Global Spatial Attention for Traffic Forecasting?
- An Empirical Audit of Input Encoders for Multi-Channel Signal Transformers
- Hybrid CNN-Transformer Based Sparse Channel Prediction for High-Mobility OTFS Systems
- Selective Learning for Deep Time Series Forecasting
- FreIE: Low-Frequency Spectral Bias in Neural Networks for Time-Series Tasks
- Time-Series Foundation Models for ISP Traffic Forecasting
- DistDF: Time-Series Forecasting Needs Joint-Distribution Wasserstein Alignment
- Quadratic Direct Forecast for Training Multi-Step Time-Series Forecast Models
- Structured Temporal Causality for Interpretable Multivariate Time Series Anomaly Detection
- Parallel BiLSTM-Transformer networks for forecasting chaotic dynamics
- Bid2X: Revealing Dynamics of Bidding Environment in Online Advertising from A Foundation Model Lens
- EMTSF:Extraordinary Mixture of SOTA Models for Time Series Forecasting
- DBLoss: Decomposition-based Loss Function for Time Series Forecasting
- Neural Architecture Search for global multi-step Forecasting of Energy Production Time Series
- NetBurst: Event-Centric Forecasting of Bursty, Intermittent Time Series
- LLM-Integrated Bayesian State Space Models for Multimodal Time-Series Forecasting
- Fusing Narrative Semantics for Financial Volatility Forecasting
- InvDec: Inverted Decoder for Multivariate Time Series Forecasting with Separated Temporal and Variate Modeling
- SynTSBench: Rethinking Temporal Pattern Learning in Deep Learning Models for Time Series
- Speculative Sampling for Parametric Temporal Point Processes
- SEMPO: Lightweight Foundation Models for Time Series Forecasting
- Abstain Mask Retain Core: Time Series Prediction by Adaptive Masking Loss with Representation Consistency
- Foundation Model Forecasts: Form and Function
- An Encode-then-Decompose Approach to Unsupervised Time Series Anomaly Detection on Contaminated Training Data--Extended Version
- Online Time Series Forecasting with Theoretical Guarantees
- Cortical-SSM: A Deep State Space Model for EEG and ECoG Motor Imagery Decoding
- Benchmarking Probabilistic Time Series Forecasting Models on Neural Activity
- Xihe: Scalable Zero-Shot Time Series Learner Via Hierarchical Interleaved Block Attention
- Fighter: Unveiling the Graph Convolutional Nature of Transformers in Time Series Modeling
- Towards Interpretable and Trustworthy Time Series Reasoning: A BlueSky Vision
- Deep learning for flash drought forecasting and interpretation
- Enhancing Time Series Forecasting through Selective Representation Spaces: A Patch Perspective
- CoRA: Covariate-Aware Adaptation of Time Series Foundation Models
- Lifting Manifolds to Mitigate Pseudo-Alignment in LLM4TS
- A Comprehensive Forecasting-Based Framework for Time Series Anomaly Detection: Benchmarking on the Numenta Anomaly Benchmark (NAB)
- Metropolis-Scale Road Network Datasets for Fine-Grained Urban Traffic Modeling
- Rethinking deep learning: linear regression remains a key benchmark in predicting terrestrial water storage
- LightSAE: Parameter-Efficient and Heterogeneity-Aware Embedding for IoT Multivariate Time Series Forecasting
- CATS-Linear: Classification Auxiliary Linear Model for Time Series Forecasting
- MoGU: Mixture-of-Gaussians with Uncertainty-based Gating for Time Series Forecasting
- HTMformer: Hybrid Time and Multivariate Transformer for Time Series Forecasting
- TimeFormer: Transformer with Attention Modulation Empowered by Temporal Characteristics for Time Series Forecasting
- TimeGazer: Temporal Modeling of Predictive Gaze Stabilization for AR Interaction
- Deciphering Invariant Feature Decoupling in Source-free Time Series Forecasting with Proxy Denoising
- Forecasting-based Biomedical Time-series Data Synthesis for Open Data and Robust AI
- Benchmarking M-LTSF: Frequency and Noise-Based Evaluation of Multivariate Long Time Series Forecasting Models
- Glocal Information Bottleneck for Time Series Imputation
- PhaseFormer: From Patches to Phases for Efficient and Effective Time Series Forecasting
- Lightweight and Data-Efficient MultivariateTime Series Forecasting using Residual-Stacked Gaussian (RS-GLinear) Architecture
- Signature-Informed Transformer for Asset Allocation
- Accuracy Law for the Future of Deep Time Series Forecasting
- RAxSS: Retrieval-Augmented Sparse Sampling for Explainable Variable-Length Medical Time Series Classification
- A Neuro-Fuzzy System for Interpretable Long-Term Stock Market Forecasting
- TimeEmb: A Lightweight Static-Dynamic Disentanglement Framework for Time Series Forecasting
- Foundation vs. Specialized Models: Evaluating Catastrophic Forgetting in Continual Time Series Forecasting
- fev-bench: A Realistic Benchmark for Time Series Forecasting
- EntroPE: Entropy-Guided Dynamic Patch Encoder for Time Series Forecasting
- ReNF: Rethinking the Design of Neural Long-Term Time Series Forecasters
- Kairos: Towards Adaptive and Generalizable Time Series Foundation Models
- Fidel-TS: A High-Fidelity Benchmark for Multimodal Time Series Forecasting
- BiHDTrans: binary hyperdimensional transformer for efficient multivariate time series classification
- Echo Flow Networks
- IndexNet: Timestamp and Variable-Aware Modeling for Time Series Forecasting
- FraudTransformer: Time-Aware GPT for Transaction Fraud Detection
- Multi-Scale Spatial-Temporal Hypergraph Network with Lead-Lag Structures for Stock Time Series Forecasting
- Channel, Trend and Periodic-Wise Representation Learning for Multivariate Long-term Time Series Forecasting
- TimeExpert: Boosting Long Time Series Forecasting with Temporal Mix of Experts
- ProtoTS: Learning Hierarchical Prototypes for Explainable Time Series Forecasting
- Beyond Model Ranking: Predictability-Aligned Evaluation for Time Series Forecasting
- From Noise to Laws: Regularized Time-Series Forecasting via Denoised Dynamic Graphs
- Revisiting Multivariate Time Series Forecasting with Missing Values
- Aurora: Towards Universal Generative Multimodal Time Series Forecasting
- Unlocking the Power of Mixture-of-Experts for Task-Aware Time Series Analytics
- Why Attention Fails: The Degeneration of Transformers into MLPs in Time Series Forecasting
- WDformer: A Wavelet-based Differential Transformer Model for Time Series Forecasting
- Lossless Compression: A New Benchmark for Time Series Model Evaluation
- Process-Informed Forecasting of Complex Thermal Dynamics in Pharmaceutical Manufacturing
- From Samples to Scenarios: A New Paradigm for Probabilistic Forecasting
- SleepLM: Natural-Language Intelligence for Human Sleep
- Spectrum Prediction in the Fractional Fourier Domain with Adaptive Filtering
- Transformer Modeling for Both Scalability and Performance in Multivariate Time Series
- GluMind: Multimodal Parallel Attention and Knowledge Retention for Robust Cross-Population Blood Glucose Forecasting
- TS-P2CL: Plug-and-Play Dual Contrastive Learning for Vision-Guided Medical Time Series Classification
- Conv-like Scale-Fusion Time Series Transformer: A Multi-Scale Representation for Variable-Length Long Time Series
- StefaLand: An Efficient Geoscience Foundation Model That Improves Dynamic Land-Surface Predictions
- MTS-DMAE: Dual-Masked Autoencoder for Unsupervised Multivariate Time Series Representation Learning
- TSGym: Design Choices for Deep Multivariate Time-Series Forecasting
- Tsururu: A Python-based Time Series Forecasting Strategies Library
- Prospective Multi-Graph Cohesion for Multivariate Time Series Anomaly Detection
- Solar Forecasting with Causality: A Graph-Transformer Approach to Spatiotemporal Dependencies
- VMDNet: Temporal Leakage-Free Variational Mode Decomposition for Electricity Demand Forecasting
- From Patterns to Predictions: A Shapelet-Based Framework for Directional Forecasting in Noisy Financial Markets
- DPANet: Dual Pyramid Attention Network for Multivariate Time Series Forecasting
- DeCoP: Enhancing Self-Supervised Time Series Representation with Dependency Controlled Pre-training
- Super-Linear: A Lightweight Pretrained Mixture of Linear Experts for Time Series Forecasting
- Bridging Past and Future: Distribution-Aware Alignment for Time Series Forecasting
- Accelerating Long-Term Molecular Dynamics with Physics-Informed Time-Series Forecasting
- FLAIRR-TS -- Forecasting LLM-Agents with Iterative Refinement and Retrieval for Time Series
- Dynamic Relational Priming Improves Transformer in Multivariate Time Series
- A Time-Series Foundation Model by Universal Delay Embedding
- A Survey of Reasoning and Agentic Systems in Time Series with Large Language Models
- SDGF: Fusing Static and Multi-Scale Dynamic Correlations for Multivariate Time Series Forecasting
- GFS: A Preemption-aware Scheduling Framework for GPU Clusters with Predictive Spot Instance Management
- ARMA Block: A CNN-Based Autoregressive and Moving Average Module for Long-Term Time Series Forecasting
- AquaCast: Urban Water Dynamics Forecasting with Precipitation-Informed Multi-Input Transformer
- S2Transformer: Scalable Structured Transformers for Global Station Weather Forecasting
- Towards Scalable and Structured Spatiotemporal Forecasting
- AdaMixT: Adaptive Weighted Mixture of Multi-Scale Expert Transformers for Time Series Forecasting
- IBN: An Interpretable Bidirectional-Modeling Network for Multivariate Time Series Forecasting with Variable Missing
- Electricity Demand and Grid Impacts of AI Data Centers: Challenges and Prospects
- Short-Term Gaze Prediction: Analysis of Individual Differences, Typical and Extreme-Case Errors
- WindFM: An Open-Source Foundation Model for Zero-Shot Wind Power Forecasting
- Neural ARFIMA model for forecasting BRIC exchange rates with long memory
- ARIES: Relation Assessment and Model Recommendation for Deep Time Series Forecasting
- VARMA-Enhanced Transformer for Time Series Forecasting
- MillGNN: Learning Multi-Scale Lead-Lag Dependencies for Multi-Variate Time Series Forecasting
- One-Embedding-Fits-All: Efficient Zero-Shot Time Series Forecasting by a Model Zoo
- Structured Basis Function Networks: Loss-Centric Multi-Hypothesis Ensembles with Controllable Diversity
- ST-Hyper: Learning High-Order Dependencies Across Multiple Spatial-Temporal Scales for Multivariate Time Series Forecasting
- StoxLSTM: A Stochastic Extended Long Short-Term Memory Network for Time Series Forecasting
- Enabling 6G Through Multi-Domain Channel Extrapolation: Opportunities and Challenges of Generative Artificial Intelligence
- Text Reinforcement for Multimodal Time Series Forecasting
- Stage-Diff: Stage-wise Long-Term Time Series Generation Based on Diffusion Models
- Filter then Attend: Improving attention-based Time Series Forecasting with Spectral Filtering
- FinCast: A Foundation Model for Financial Time-Series Forecasting
- LLM-Driven Self-Refinement for Embodied Drone Task Planning
- STDiff: A State Transition Diffusion Framework for Time Series Imputation in Industrial Systems
- PAX-TS: Model-agnostic multi-granular explanations for time series forecasting via localized perturbations
- Frozen in Time: Parameter-Efficient Time Series Transformers via Reservoir-Induced Feature Expansion and Fixed Random Dynamics
- Structured Agentic Workflows for Financial Time-Series Modeling with LLMs and Reflective Feedback
- PENGUIN: Enhancing Transformer with Periodic-Nested Group Attention for Long-term Time Series Forecasting
- HRS: Hybrid Representation Framework with Scheduling Awareness for Time Series Forecasting in Crowdsourced Cloud-Edge Platforms
- Causally-Guided Pairwise Transformer -- Towards Foundational Digital Twins in Process Industry
- Bi-Axial Transformers: Addressing the Increasing Complexity of EHR Classification
- Estimating Covariance for Global Minimum Variance Portfolio: A Decision-Focused Learning Approach
- Wavelet Mixture of Experts for Time Series Forecasting
- Load Forecasting on A Highly Sparse Electrical Load Dataset Using Gaussian Interpolation
- Fre-CW: Targeted Attack on Time Series Forecasting using Frequency Domain Loss
- Semantic-Enhanced Time-Series Forecasting via Large Language Models
- Weather-Driven Agricultural Decision-Making Using Digital Twins Under Imperfect Conditions
- QuiZSF: An efficient data-model interaction framework for zero-shot time-series forecasting
- Comparative Analysis of Time Series Foundation Models for Demographic Forecasting: Enhancing Predictive Accuracy in US Population Dynamics
- Time series forecasting of train axle fatigue crack acoustic emission signals by integrating multi-head attention mechanism into DLinear model
- From Values to Tokens: An LLM-Driven Framework for Context-aware Time Series Forecasting via Symbolic Discretization
- PRISM: Lightweight Multivariate Time-Series Classification through Symmetric Multi-Resolution Convolutional Layers
- PA-RNet: Perturbation-Aware Reasoning Network for Multimodal Time Series Forecasting
- Empowering Time Series Forecasting with LLM-Agents
- FeDaL: Federated Dataset Learning for Time Series Foundation Models
- DeepKoopFormer: A Koopman Enhanced Transformer Based Architecture for Time Series Forecasting
- JustDense: Just using Dense instead of Sequence Mixer for Time Series analysis
- User Trajectory Prediction Unifying Global and Local Temporal Information
- HGTS-Former: Hierarchical HyperGraph Transformer for Multivariate Time Series Analysis
- KANMixer: Can KAN Serve as a New Modeling Core for Long-term Time Series Forecasting?
- Signals, Concepts, and Laws: Toward Universal, Explainable Time-Series Forecasting
- TriP-LLM: A Tri-Branch Patch-wise Large Language Model Framework for Time-Series Anomaly Detection
- OKG-LLM: Aligning Ocean Knowledge Graph with Observation Data via LLMs for Global Sea Surface Temperature Prediction
- DualSG: A Dual-Stream Explicit Semantic-Guided Multivariate Time Series Forecasting Framework
Discussions
Related