Are Data Embeddings effective in time series forecasting?

2025/05/27 by Reza Nematirad, Nematirad, Reza, Anil Pahwa +3
Computer Science · Decision Sciences · #FOS: Computer and information sciences #Forecasting Techniques and Applications #Machine Learning (cs.LG) #Stock Market Forecasting Methods #Time Series Analysis and Forecasting

paper · pdf · doi:10.48550/arxiv.2505.20716

openalex publication_date 2025/05/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Time series forecasting plays a crucial role in many real-world applications, and numerous complex forecasting models have been proposed in recent years. Despite their architectural innovations, most state-of-the-art models report only marginal improvements -- typically just a few thousandths in standard error metrics. These models often incorporate complex data embedding layers to transform raw inputs into higher-dimensional representations to enhance accuracy. But are data embedding techniques actually effective in time series forecasting? Through extensive ablation studies across fifteen state-of-the-art models and four benchmark datasets, we find that removing data embedding layers from many state-of-the-art models does not degrade forecasting performance. In many cases, it improves both accuracy and computational efficiency. The gains from removing embedding layers often exceed the performance differences typically reported between competing models. Code available at: https://github.com/neuripsdataembedidng/DataEmbedding

Citations

• Times2D: Multi-Period Decomposition and Derivative Mapping for General Time Series Forecasting
• VarDrop: Enhancing Training Efficiency by Reducing Variate Redundancy in Periodic Time Series Forecasting
• EDformer: Embedded Decomposition Transformer for Interpretable Multivariate Time Series Predictions
• LiNo: Advancing Recursive Residual Decomposition of Linear and Nonlinear Patterns for Robust Time Series Forecasting
• TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting
• SOFTS: Efficient Multivariate Time Series Forecasting with Series-Core Fusion
• TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables
• ContiFormer: Continuous-Time Transformer for Irregular Time Series Modeling
• Minusformer: Improving Time Series Forecasting by Progressively Learning Residuals
• iTransformer: Inverted Transformers Are Effective for Time Series Forecasting
• Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
• A Time Series is Worth 64 Words: Long-term Forecasting with Transformers
• ETSformer: Exponential Smoothing Transformers for Time-series Forecasting

Related

• Integrating Time Series into LLMs via Multi-layer Steerable Embedding Fusion for Enhanced Forecasting
• Decomposing the Time Series Forecasting Pipeline: A Modular Approach for Time Series Representation, Information Extraction, and Projection
• Are Language Models Actually Useful for Time Series Forecasting?
• What Matters in Deep Learning for Time Series Forecasting?
• Improving Accuracy Without Losing Interpretability: A ML Approach for Time Series Forecasting