FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting

2022/01/30 by Tian Zhou, Ziqing Ma, Zhou, Tian +9 · 187 citations
Computer Science · Decision Sciences · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications #Stock Market Forecasting Methods #Time Series Analysis and Forecasting

paper · pdf · doi:10.48550/arxiv.2201.12740

openalex publication_date 2022/01/30 · openalex created_date 2022/05/05 · openalex updated_date 2026/07/28

Abstract

Although Transformer-based methods have significantly improved state-of-the-art results for long-term series forecasting, they are not only computationally expensive but more importantly, are unable to capture the global view of time series (e.g. overall trend). To address these problems, we propose to combine Transformer with the seasonal-trend decomposition method, in which the decomposition method captures the global profile of time series while Transformers capture more detailed structures. To further enhance the performance of Transformer for long-term prediction, we exploit the fact that most time series tend to have a sparse representation in well-known basis such as Fourier transform, and develop a frequency enhanced Transformer. Besides being more effective, the proposed method, termed as Frequency Enhanced Decomposed Transformer (\bf FEDformer), is more efficient than standard Transformer with a linear complexity to the sequence length. Our empirical studies with six benchmark datasets show that compared with state-of-the-art methods, FEDformer can reduce prediction error by 14.8% and 22.6% for multivariate and univariate time series, respectively. Code is publicly available at https://github.com/MAZiqing/FEDformer.

Cited by

• What Matters in Deep Learning for Time Series Forecasting?
• QuantFlow: A Federated Mamba-Based Post-Transformer Foundation Model for Time-Series Forecasting
• Differencing the Diffusion Trajectory toward Uncertain Components for Time Series Forecasting
• DSTFView: Multi-View Cloud-Edge Workload Forecasting with Dual-Input Spatio-Temporal-Frequency Modeling
• Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting
• Transformer-Based Modeling of User Interaction Sequences for Dwell Time Prediction in Human-Computer Interfaces
• Dynamic Attention (DynAttn): Interpretable High-Dimensional Spatio-Temporal Forecasting (with Application to Conflict Fatalities)
• DecoKAN: Interpretable Decomposition for Forecasting Cryptocurrency Market Dynamics
• KOSS: Kalman-Optimal Selective State Spaces for Long-Term Sequence Modeling
• TwinFormer: A Dual-Level Transformer for Long-Sequence Time-Series Forecasting
• Neural CDEs as Correctors for Learned Time Series Models
• Contrastive Time Series Forecasting with Anomalies
• DB2-TransF: All You Need Is Learnable Daubechies Wavelets for Time Series Forecasting
• Adaptive Information Routing for Multimodal Time Series Forecasting
• Weighted Contrastive Learning for Anomaly-Aware Time-Series Forecasting
• Adaptive Normalization Mamba with Multi Scale Trend Decomposition and Patch MoE Encoding
• OSM+: Billion-Level Open Street Map Data Processing System for City-wide Experiments
• IdealTSF: Can Non-Ideal Data Contribute to Enhancing the Performance of Time Series Forecasting Models?
• Smart Timing for Mining: A Deep Learning Framework for Bitcoin Hardware ROI Prediction
• STELLA: Guiding Large Language Models for Time Series Forecasting with Semantic Abstractions
• ESACT: An End-to-End Sparse Accelerator for Compute-Intensive Transformers via Local Similarity
• A Multi-View Multi-Timescale Hypergraph-Empowered Spatiotemporal Framework for EV Charging Forecasting
• DPWMixer: Dual-Path Wavelet Mixer for Long-Term Time Series Forecasting
• FiCoTS: Fine-to-Coarse LLM-Enhanced Hierarchical Cross-Modality Interaction for Time Series Forecasting
• TARFVAE: Efficient One-Step Generative Time Series Forecasting via TARFLOW based VAE
• RED-F: Reconstruction-Elimination based Dual-stream Contrastive Forecasting for Multivariate Time Series Anomaly Prediction
• MSTN: Fast and Efficient Multivariate Time Series Prediction Model
• Temporal-Visual Semantic Alignment: A Unified Architecture for Transferring Spatial Priors from Vision Models to Zero-Shot Temporal Tasks
• SimDiff: Simpler Yet Better Diffusion Model for Time Series Point Forecasting
• WaveTuner: Comprehensive Wavelet Subband Tuning for Time Series Forecasting
• TimePre: Bridging Accuracy, Efficiency, and Stability in Probabilistic Time-Series Forecasting
• PeriodNet: Boosting the Potential of Attention Mechanism for Time Series Forecasting
• Partial multivariate transformer as a tool for cryptocurrencies time series prediction
• Unmasking Airborne Threats: Guided-Transformers for Portable Aerosol Mass Spectrometry
• A FEDformer-Based Hybrid Framework for Anomaly Detection and Risk Forecasting in Financial Time Series
• Adapformer: Adaptive Channel Management for Multivariate Time Series Forecasting
• PAST: A Primary-Auxiliary Spatio-Temporal Network for Traffic Time Series Imputation
• ReCast: Reliability-aware Codebook Assisted Lightweight Time Series Forecasting
• FreDN: Spectral Disentanglement for Time Series Forecasting via Learnable Frequency Decomposition
• VitalBench: A Rigorous Multi-Center Benchmark for Long-Term Vital Sign Prediction in Intraoperative Care
• Beyond MSE: Ordinal Cross-Entropy for Probabilistic Time Series Forecasting
• Trend-Aware Multi-Task Learning for Short-Term Energy Forecasting
• OmniFuser: Adaptive Multimodal Fusion for Service-Oriented Predictive Maintenance
• EMAformer: Enhancing Transformer through Embedding Armor for Time Series Forecasting
• IBMA: An Imputation-Based Mixup Augmentation Using Self-Supervised Learning for Time Series Data
• HN-MVTS: HyperNetwork-based Multivariate Time Series Forecasting
• The Few Govern the Many:Unveiling Few-Layer Dominance for Time Series Models
• DeepBooTS: Dual-Stream Residual Boosting for Drift-Resilient Time-Series Forecasting
• COTN: A Chaotic Oscillatory Transformer Network for Complex Volatile Systems under Extreme Conditions
• Predicting the Future by Retrieving the Past
• AWEMixer: Adaptive Wavelet-Enhanced Mixer Network for Long-Term Time Series Forecasting
• ForecastGAN: A Decomposition-Based Adversarial Framework for Multi-Horizon Time Series Forecasting
• FlowNet: Modeling Dynamic Spatio-Temporal Systems via Flow Propagation
• Hydra: Dual Exponentiated Memory for Multivariate Time Series Analysis
• Effective Series Decomposition and Components Learning for Time Series Generation
• Temporal Fusion Transformer for Multi-Horizon Probabilistic Forecasting of Weekly Retail Sales
• TS-Reasoner: Aligning Time Series Foundation Models with LLM Reasoning
• Selective Learning for Deep Time Series Forecasting
• FreIE: Low-Frequency Spectral Bias in Neural Networks for Time-Series Tasks
• OneCast: Structured Decomposition and Modular Generation for Cross-Domain Time Series Forecasting
• Bid2X: Revealing Dynamics of Bidding Environment in Online Advertising from A Foundation Model Lens
• EMTSF:Extraordinary Mixture of SOTA Models for Time Series Forecasting
• How AI Forecasts AI Jobs: Benchmarking LLM Predictions of Labor Market Changes
• DBLoss: Decomposition-based Loss Function for Time Series Forecasting
• Neural Architecture Search for global multi-step Forecasting of Energy Production Time Series
• NetBurst: Event-Centric Forecasting of Bursty, Intermittent Time Series
• InvDec: Inverted Decoder for Multivariate Time Series Forecasting with Separated Temporal and Variate Modeling
• SynTSBench: Rethinking Temporal Pattern Learning in Deep Learning Models for Time Series
• SEMPO: Lightweight Foundation Models for Time Series Forecasting
• Abstain Mask Retain Core: Time Series Prediction by Adaptive Masking Loss with Representation Consistency
• Cortical-SSM: A Deep State Space Model for EEG and ECoG Motor Imagery Decoding
• Fighter: Unveiling the Graph Convolutional Nature of Transformers in Time Series Modeling
• Towards Interpretable and Trustworthy Time Series Reasoning: A BlueSky Vision
• Tackling Time-Series Forecasting Generalization via Mitigating Concept Drift
• Enhancing Time Series Forecasting through Selective Representation Spaces: A Patch Perspective
• A Physics Prior-Guided Dual-Stream Attention Network for Motion Prediction of Elastic Bragg Breakwaters
• Lifting Manifolds to Mitigate Pseudo-Alignment in LLM4TS
• A Comprehensive Forecasting-Based Framework for Time Series Anomaly Detection: Benchmarking on the Numenta Anomaly Benchmark (NAB)
• WaveletDiff: Multilevel Wavelet Diffusion For Time Series Generation
• SVTime: Small Time Series Forecasting Models Informed by "Physics" of Large Vision Model Forecasters
• IF-D: A High-Frequency, General-Purpose Inertial Foundation Dataset for Self-Supervised Learning
• Augur: Modeling Covariate Causal Associations in Time Series via Large Language Models
• Inner-Instance Normalization for Time Series Forecasting
• MLLM4TS: Leveraging Vision and Multimodal Language Models for General Time-Series Analysis
• HTMformer: Hybrid Time and Multivariate Transformer for Time Series Forecasting
• TimeFormer: Transformer with Attention Modulation Empowered by Temporal Characteristics for Time Series Forecasting
• StruSR: Structure-Aware Symbolic Regression with Physics-Informed Taylor Guidance
• TimeGazer: Temporal Modeling of Predictive Gaze Stabilization for AR Interaction
• TelecomTS: A Multi-Modal Observability Dataset for Time Series and Language Analysis
• Deciphering Invariant Feature Decoupling in Source-free Time Series Forecasting with Proxy Denoising
• The New Quant: A Survey of Large Language Models in Financial Prediction and Trading
• Rivaling Transformers: Multi-Scale Structured State-Space Mixtures for Agentic 6G O-RAN
• Wavelet Predictive Representations for Non-Stationary Reinforcement Learning
• Benchmarking M-LTSF: Frequency and Noise-Based Evaluation of Multivariate Long Time Series Forecasting Models
• PhaseFormer: From Patches to Phases for Efficient and Effective Time Series Forecasting
• Modeling Time Series Dynamics with Fourier Ordinary Differential Equations
• Signature-Informed Transformer for Asset Allocation
• TimeEmb: A Lightweight Static-Dynamic Disentanglement Framework for Time Series Forecasting
• How Foundational are Foundation Models for Time Series Forecasting?
• EntroPE: Entropy-Guided Dynamic Patch Encoder for Time Series Forecasting
• DSAT-HD: Dual-Stream Adaptive Transformer with Hybrid Decomposition for Multivariate Time Series Forecasting
• BiHDTrans: binary hyperdimensional transformer for efficient multivariate time series classification
• IndexNet: Timestamp and Variable-Aware Modeling for Time Series Forecasting
• A Self-Adaptive Frequency Domain Network for Continuous Intraoperative Hypotension Prediction
• Graph Neural Networks with Diversity-aware Neighbor Selection and Dynamic Multi-scale Fusion for Multivariate Time Series Forecasting
• Channel, Trend and Periodic-Wise Representation Learning for Multivariate Long-term Time Series Forecasting
• TimeExpert: Boosting Long Time Series Forecasting with Temporal Mix of Experts
• Beyond Model Ranking: Predictability-Aligned Evaluation for Time Series Forecasting
• From Noise to Laws: Regularized Time-Series Forecasting via Denoised Dynamic Graphs
• Unlocking the Power of Mixture-of-Experts for Task-Aware Time Series Analytics
• ChaosNexus: A Foundation Model for Universal Chaotic System Forecasting with Multi-scale Representations
• Conditionally Whitened Generative Models for Probabilistic Time Series Forecasting
• WDformer: A Wavelet-based Differential Transformer Model for Time Series Forecasting
• VFEM: Visual Feature Empowered Multivariate Time Series Forecasting with Cross-Modal Fusion
• CoSupFormer : A Contrastive Supervised learning approach for EEG signal Classification
• Spectrum Prediction in the Fractional Fourier Domain with Adaptive Filtering
• TS-P²CL: Plug-and-Play Dual Contrastive Learning for Vision-Guided Medical Time Series Classification
• Conv-like Scale-Fusion Time Series Transformer: A Multi-Scale Representation for Variable-Length Long Time Series
• MTS-DMAE: Dual-Masked Autoencoder for Unsupervised Multivariate Time Series Representation Learning
• TSGym: Design Choices for Deep Multivariate Time-Series Forecasting
• From Patterns to Predictions: A Shapelet-Based Framework for Directional Forecasting in Noisy Financial Markets
• GateTS: Versatile and Efficient Forecasting via Attention-Inspired routed Mixture-of-Experts
• DPANet: Dual Pyramid Attention Network for Multivariate Time Series Forecasting
• Super-Linear: A Lightweight Pretrained Mixture of Linear Experts for Time Series Forecasting
• TFMAdapter: Lightweight Instance-Level Adaptation of Foundation Models for Forecasting with Covariates
• Bridging Past and Future: Distribution-Aware Alignment for Time Series Forecasting
• FLAIRR-TS -- Forecasting LLM-Agents with Iterative Refinement and Retrieval for Time Series
• NHITS: Neural Hierarchical Interpolation for Time Series Forecasting
• Detecting Multilevel Manipulation from Limit Order Book via Cascaded Contrastive Representation Learning
• GFS: A Preemption-aware Scheduling Framework for GPU Clusters with Predictive Spot Instance Management
• ARMA Block: A CNN-Based Autoregressive and Moving Average Module for Long-Term Time Series Forecasting
• Unsupervised Multi-Attention Meta Transformer for Rotating Machinery Fault Diagnosis
• DeepAries: Adaptive Rebalancing Interval Selection for Enhanced Portfolio Selection
• S²Transformer: Scalable Structured Transformers for Global Station Weather Forecasting
• Towards Scalable and Structured Spatiotemporal Forecasting
• AdaMixT: Adaptive Weighted Mixture of Multi-Scale Expert Transformers for Time Series Forecasting
• WindFM: An Open-Source Foundation Model for Zero-Shot Wind Power Forecasting
• ARIES: Relation Assessment and Model Recommendation for Deep Time Series Forecasting
• Real-E: A Foundation Benchmark for Advancing Robust and Generalizable Electricity Forecasting
• VARMA-Enhanced Transformer for Time Series Forecasting
• MillGNN: Learning Multi-Scale Lead-Lag Dependencies for Multi-Variate Time Series Forecasting
• One-Embedding-Fits-All: Efficient Zero-Shot Time Series Forecasting by a Model Zoo
• Data-Efficient Psychiatric Disorder Detection via Self-supervised Learning on Frequency-enhanced Brain Networks
• Structured Basis Function Networks: Loss-Centric Multi-Hypothesis Ensembles with Controllable Diversity
• STA-GANN: A Valid and Generalizable Spatio-Temporal Kriging Approach
• ST-Hyper: Learning High-Order Dependencies Across Multiple Spatial-Temporal Scales for Multivariate Time Series Forecasting
• StoxLSTM: A Stochastic Extended Long Short-Term Memory Network for Time Series Forecasting
• Integrating Time Series into LLMs via Multi-layer Steerable Embedding Fusion for Enhanced Forecasting
• Text Reinforcement for Multimodal Time Series Forecasting
• Quantum-Optimized Selective State Space Model for Efficient Time Series Prediction
• Filter then Attend: Improving attention-based Time Series Forecasting with Spectral Filtering
• HierCVAE: Hierarchical Attention-Driven Conditional Variational Autoencoders for Multi-Scale Temporal Modeling
• Frozen in Time: Parameter-Efficient Time Series Transformers via Reservoir-Induced Feature Expansion and Fixed Random Dynamics
• PENGUIN: Enhancing Transformer with Periodic-Nested Group Attention for Long-term Time Series Forecasting
• HRS: Hybrid Representation Framework with Scheduling Awareness for Time Series Forecasting in Crowdsourced Cloud-Edge Platforms
• CC-Time: Cross-Model and Cross-Modality Time Series Forecasting
• Scale-Disentangled spatiotemporal Modeling for Long-term Traffic Emission Forecasting
• Self-Supervised Temporal Super-Resolution of Energy Data using Generative Adversarial Transformer
• TimeMKG: Knowledge-Infused Causal Reasoning for Multivariate Time Series Modeling
• Wavelet Mixture of Experts for Time Series Forecasting
• Fre-CW: Targeted Attack on Time Series Forecasting using Frequency Domain Loss
• Stationarity Exploration for Multivariate Time Series Forecasting
• QuiZSF: An efficient data-model interaction framework for zero-shot time-series forecasting
• From Values to Tokens: An LLM-Driven Framework for Context-aware Time Series Forecasting via Symbolic Discretization
• EnergyPatchTST: Multi-scale Time Series Transformers with Uncertainty Estimation for Energy Forecasting
• PriceFM: Foundation Model for Probabilistic Electricity Price Forecasting
• PRISM: Lightweight Multivariate Time-Series Classification through Symmetric Multi-Resolution Convolutional Layers
• VisionTS++: Cross-Modal Time Series Foundation Model with Continual Pre-trained Vision Backbones
• T3Time: Tri-Modal Time Series Forecasting via Adaptive Multi-Head Alignment and Residual Fusion
• PA-RNet: Perturbation-Aware Reasoning Network for Multimodal Time Series Forecasting
• DP-GPT4MTS: Dual-Prompt Large Language Model for Textual-Numerical Time Series Forecasting
• FeDaL: Federated Dataset Learning for Time Series Foundation Models
• DeepKoopFormer: A Koopman Enhanced Transformer Based Architecture for Time Series Forecasting
• HGTS-Former: Hierarchical HyperGraph Transformer for Multivariate Time Series Analysis
• KANMixer: Can KAN Serve as a New Modeling Core for Long-term Time Series Forecasting?
• Frequency-Constrained Learning for Long-Term Forecasting
• Signals, Concepts, and Laws: Toward Universal, Explainable Time-Series Forecasting
• Kronos: A Foundation Model for the Language of Financial Markets
• KFS: KAN based adaptive Frequency Selection learning architecture for long term time series forecasting
• Multi-grained spatial-temporal feature complementarity for accurate online cellular traffic prediction
• XFMNet: Decoding Cross-Site and Nonstationary Water Patterns via Stepwise Multimodal Fusion for Long-Term Water Quality Forecasting
• OKG-LLM: Aligning Ocean Knowledge Graph with Observation Data via LLMs for Global Sea Surface Temperature Prediction
• Predicting Large-scale Urban Network Dynamics with Energy-informed Graph Neural Diffusion
• Are Transformers Effective for Time Series Forecasting?
• PAF-Net: Phase-Aligned Frequency Decoupling Network for Multi-Process Manufacturing Quality Prediction
• Gems: Group Emotion Profiling Through Multimodal Situational Understanding
• DualSG: A Dual-Stream Explicit Semantic-Guided Multivariate Time Series Forecasting Framework

Related

• ETSformer: Exponential Smoothing Transformers for Time-series Forecasting
• Scaleformer: Iterative Multi-scale Refining Transformers for Time Series Forecasting
• Multi-resolution Time-Series Transformer for Long-term Forecasting
• Dateformer: Time-modeling Transformer for Longer-term Series Forecasting
• EDformer: Embedded Decomposition Transformer for Interpretable Multivariate Time Series Predictions