2023/05/24 by Yushu Chen, Chen, Yushu, Shengzhuo Liu +9 · 1 citation
Computer Science · Decision Sciences · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications #Stock Market Forecasting Methods #Time Series Analysis and Forecasting
paper · pdf · doi:10.48550/arxiv.2305.14649
openalex publication_date 2023/05/24 · openalex created_date 2023/05/27 · openalex updated_date 2026/07/28
In order to enhance the performance of Transformer models for long-term multivariate forecasting while minimizing computational demands, this paper introduces the Joint Time-Frequency Domain Transformer (JTFT). JTFT combines time and frequency domain representations to make predictions. The frequency domain representation efficiently extracts multi-scale dependencies while maintaining sparsity by utilizing a small number of learnable frequencies. Simultaneously, the time domain (TD) representation is derived from a fixed number of the most recent data points, strengthening the modeling of local relationships and mitigating the effects of non-stationarity. Importantly, the length of the representation remains independent of the input sequence length, enabling JTFT to achieve linear computational complexity. Furthermore, a low-rank attention layer is proposed to efficiently capture cross-dimensional dependencies, thus preventing performance degradation resulting from the entanglement of temporal and channel-wise modeling. Experimental results on six real-world datasets demonstrate that JTFT outperforms state-of-the-art baselines in predictive performance.