DualPathTST: A Dual-Pathway PatchTST Utilizing Cross-Variable Dependency and De-Stationary for Time Series Forecasting

In recent years, Transformer-like models have increasingly underscored their importance in Time Series Forecasting (TSF), playing a pivotal role across various fields. However, their attention mechanisms never consider the dependencies across variables, which can diminish the predictive performance. Moreover, these models overlook the inherent non-stationarity of real-world scenarios. To address these challenges, a Dual-Pathway PatchTST model, namely DualPathTST, is developed, where a de-stationary attention mechanism is employed into the original pathway, while a convolutional pathway is designed to capture the cross-variable dependencies. Then, a gated fusion mechanism is introduced to reconcile information from two pathways, which would dynamically integrate internal relationships and cross-variable dependencies. To effectively mitigate internal covariate shifts and enhance the model’s stability, a batch normalization layer is strategically incorporated into the output module. Extensive testing on multiple datasets, including ETT, Exchanges, Electricity, Weather, and Traffic, has conclusively demonstrated that DualPathTST not only significantly outperforms state-of-the-art models, with an average improvement of 4.6%, but also provides an innovative solution to the problem of capturing dependencies across variables in TSF models.