2025/01/06 by Liyang Qin, Xiaoli Wang, Qin, Liyang +6
Computer Science · Economics, Econometrics and Finance · #Anomaly Detection Techniques and Applications #Complex Systems and Time Series Analysis #FOS: Computer and information sciences #I.2.4 #I.2.6 #Machine Learning (cs.LG) #Time Series Analysis and Forecasting
paper · pdf · doi:10.48550/arxiv.2501.03284
openalex publication_date 2025/01/06 · openalex created_date 2025/01/09 · openalex updated_date 2026/07/28
Among the existing Transformer-based multivariate time series forecasting methods, iTransformer, which treats each variable sequence as a token and only explicitly extracts cross-variable dependencies, and PatchTST, which adopts a channel-independent strategy and only explicitly extracts cross-time dependencies, both significantly outperform most Channel-Dependent Transformer that simultaneously extract cross-time and cross-variable dependencies. This indicates that existing Transformer-based multivariate time series forecasting methods still struggle to effectively fuse these two types of information. We attribute this issue to the dynamic time lags in the causal relationships between different variables. Therefore, we propose a new multivariate time series forecasting Transformer, Sensorformer, which first compresses the global patch information and then simultaneously extracts cross-variable and cross-time dependencies from the compressed representations. Sensorformer can effectively capture the correct inter-variable correlations and causal relationships, even in the presence of dynamic causal lags between variables, while also reducing the computational complexity of pure cross-patch self-attention from O(D2 ⋅ Patch_num2 ⋅ d_model) to O(D2 ⋅ Patch_num ⋅ d_model). Extensive comparative and ablation experiments on 9 mainstream real-world multivariate time series forecasting datasets demonstrate the superiority of Sensorformer. The implementation of Sensorformer, following the style of the Time-series-library and scripts for reproducing the main results, is publicly available at https://github.com/BigYellowTiger/Sensorformer