Journal Home > Volume 28 , Issue 1
Tsinghua Science and Technology 2023, 28(1): 141-149 https://doi.org/10.26599/TST.2021.9010081
Open Access | Issue | Published: 21 July 2022

Multivariate Time Series Forecasting with Transfer Entropy Graph

Show Author's Information Ziheng Duan1,  Haoyan Xu2,  Yida Huang2,  Jie Feng2,  Yueyang Wang1( )
School of Big Data and Software Engineering, Chongqing University, Chongqing 401331, China
College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China
Keywords:
Multivariate Time Series (MTS) forecasting, neural Granger causality graph, Transfer Entropy (TE)
Cite this article:
Duan Z, Xu H, Huang Y, et al. Multivariate Time Series Forecasting with Transfer Entropy Graph. Tsinghua Science and Technology, 2023, 28(1): 141-149. https://doi.org/10.26599/TST.2021.9010081
Download citation
EndNote(RIS)
BibTeX

648

Views

57

Downloads

Citations

7

Crossref

7

WoS

8

Scopus

0

CSCD

Abstract Full text About this article

Multivariate Time Series (MTS) forecasting is an essential problem in many fields. Accurate forecasting results can effectively help in making decisions. To date, many MTS forecasting methods have been proposed and widely applied. However, these methods assume that the predicted value of a single variable is affected by all other variables, ignoring the causal relationship among variables. To address the above issue, we propose a novel end-to-end deep learning model, termed graph neural network with neural Granger causality, namely CauGNN, in this paper. To characterize the causal information among variables, we introduce the neural Granger causality graph in our model. Each variable is regarded as a graph node, and each edge represents the casual relationship between variables. In addition, convolutional neural network filters with different perception scales are used for time series feature extraction, to generate the feature of each node. Finally, the graph neural network is adopted to tackle the forecasting problem of the graph structure generated by the MTS. Three benchmark datasets from the real world are used to evaluate the proposed CauGNN, and comprehensive experiments show that the proposed method achieves state-of-the-art results in the MTS forecasting task.


menu
Abstract
Full text
Outline
About this article

Multivariate Time Series Forecasting with Transfer Entropy Graph

Show Author's information Ziheng Duan1, Haoyan Xu2, Yida Huang2, Jie Feng2, Yueyang Wang1( )
School of Big Data and Software Engineering, Chongqing University, Chongqing 401331, China
College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China

Abstract

Multivariate Time Series (MTS) forecasting is an essential problem in many fields. Accurate forecasting results can effectively help in making decisions. To date, many MTS forecasting methods have been proposed and widely applied. However, these methods assume that the predicted value of a single variable is affected by all other variables, ignoring the causal relationship among variables. To address the above issue, we propose a novel end-to-end deep learning model, termed graph neural network with neural Granger causality, namely CauGNN, in this paper. To characterize the causal information among variables, we introduce the neural Granger causality graph in our model. Each variable is regarded as a graph node, and each edge represents the casual relationship between variables. In addition, convolutional neural network filters with different perception scales are used for time series feature extraction, to generate the feature of each node. Finally, the graph neural network is adopted to tackle the forecasting problem of the graph structure generated by the MTS. Three benchmark datasets from the real world are used to evaluate the proposed CauGNN, and comprehensive experiments show that the proposed method achieves state-of-the-art results in the MTS forecasting task.

Keywords: Multivariate Time Series (MTS) forecasting, neural Granger causality graph, Transfer Entropy (TE)

References(41)

[1]
Z. H. Duan, H. Y. Xu, Y. Y. Wang, Y. D. Huang, A. N. Ren, Z. B. Xu, Y. Z. Sun, and W. Wang, Multivariate time series classification with hierarchical variational graph pooling, arXiv preprint arXiv: 2010.05649, 2020.
Google Scholar
[2]
H. Chen, S. Feng, X. Pei, Z. Zhang, and D. Yao, Dangerous driving behavior recognition and prevention using an autoregressive time-series model, Tsinghua Science and Technology, vol. 22, no. 6, pp. 682–690, 2017.
DOI Google Scholar
[3]
Y. S. Liu, C. H. Yang, K. K. Huang, and W. H. Gui, Nonferrous metals price forecasting based on variational mode decomposition and LSTM network, Knowl. Based Syst., vol. 188, p. 105006, 2020.
DOI Google Scholar
[4]
L. Yu, S. Dong, M. Lu, and J. Wang, LSTM based reserve prediction for bank outlets, Tsinghua Science and Technology, 2018, vol. 24, no. 1, pp. 77–85, 2018.
DOI Google Scholar
[5]
Y. Y. Wang, Z. H. Duan, Y. D. Huang, H. Y. Xu, J. Feng, and A. N. Ren, MTHetGNN: A heterogeneous graph embedding framework for multivariate time series forecasting, Pattern Recognit. Lett., vol. 153, pp. 151–158, 2022.
DOI Google Scholar
[6]
G. E. P. Box, G. M. Jenkins, G. C. Reinsel, and G. M. Ljung, Time Series Analysis: Forecasting and Control, 4th Edition, Hoboken, NJ, USA: John Wiley & Sons, 2016.
[7]
J. D. Hamilton, Time Series Analysis. Princeton, NJ, USA: Princeton University Press, 1994.
DOI
[8]
H. Lütkepohl, New Introduction to Multiple Time Series Analysis. Berlin, Germany: Springer, 2005.
DOI
[9]
A. Tokgöz and G. Ünal, A RNN based time series approach for forecasting Turkish electricity load, in Proc. 2018 26th Signal Processing and Communications Applications Conf. (SIU), Izmir, Turkey, 2018, pp. 1–4.
DOI Google Scholar
[10]
J. L. Elman, Finding structure in time, Cogn. Sci., vol. 14, no. 2, pp. 179–211, 1990.
DOI Google Scholar
[11]
S. Hochreiter and J. Schmidhuber, Long short-term memory, Neural Comput., vol. 9, no. 8, pp. 1735–1780, 1997.
DOI Google Scholar
[12]
J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, Empirical evaluation of gated recurrent neural networks on sequence modeling, arXiv preprint arXiv: 1412.3555, 2014.
Google Scholar
[13]
G. K. Lai, W. C. Chang, Y. M. Yang, and H. X. Liu, Modeling long- and short-term temporal patterns with deep neural networks, in Proc. the 41st Int. ACM SIGIR Conf. on Research & Development in Information Retrieval, Ann Arbor, MI, USA, 2017, pp. 95–104.
DOI Google Scholar
[14]
Y. F. Zhou, Z. H. Duan, H. Y. Xu, J. Feng, A. N. Ren, Y. Y. Wang, and X. Q. Wang, Parallel extraction of long-term trends and short-term fluctuation framework for multivariate time series forecasting, arXiv preprint arXiv: 2008.07730, 2020.
Google Scholar
[15]
C. W. J. Granger, Investigating causal relations by econometric models and cross-spectral methods, Econometrica, vol. 37, no. 3, pp. 424–438, 1969.
DOI Google Scholar
[16]
G. Kirchgässner, J. Wolters, and U. Hassler, Granger causality, in Introduction to Modern Time Series Analysis, G. Kirchgässner, J. Wolters, and U. Hassler, eds. Berlin, Germany: Springer, 2013, pp. 95–125.
DOI
[17]
T. Bossomaier, L. Barnett, M. Harré, and J. T. Lizier, Transfer entropy, in An Introduction to Transfer Entropy, T. Bossomaier, L. Barnett, M. Harré, and J. T. Lizier, eds. Cham, Switzerland: Springer, 2016, pp. 65–95.
DOI
[18]
T. Dimpfl and F. J. Peter, Using transfer entropy to measure information flows between financial markets, Stud. Nonlinear Dyn. Econom., vol. 17, no. 1, pp. 85–102, 2013.
DOI Google Scholar
[19]
T. Q. Tung, T. Ryu, K. H. Lee, and D. Lee, Inferring gene regulatory networks from microarray time series data using transfer entropy, in Proc. 20th IEEE Int. Symp. on Computer-Based Medical Systems (CBMS’07), Maribor, Slovenia, 2007, pp. 383–388.
DOI Google Scholar
[20]
M. Bauer, J. W. Cox, M. H. Caveness, J. J. Downs, and N. F. Thornhill, Finding the direction of disturbance propagation in a chemical process using transfer entropy, IEEE Trans. Control Syst. Technol., vol. 15, no. 1, pp. 12–21, 2007.
DOI Google Scholar
[21]
F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, The graph neural network model, IEEE Trans. Neural Netw., vol. 20, no. 1, pp. 61–80, 2009.
DOI Google Scholar
[22]
Y. Y Wang, Z. H. Duan, B. B. Liao, F. Wu, and Y. T. Zhuang, Heterogeneous attributed network embedding with graph convolutional networks, in Proc. the AAAI Conf. on Artificial Intelligence, Palo Alto, CA, USA, 2019, pp. 10061&10062.
DOI Google Scholar
[23]
Z. H. Duan, Y. Y. Wang, W. H. Ye, Z. X. Feng, Q. L. Fan, and X. H. Li, Connecting latent relationships over heterogeneous attributed network for recommendation, arXiv preprint arXiv: 2103.05749, 2021.
Google Scholar
[24]
J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, Neural message passing for quantum chemistry, in Proc. the 34th Int. Conf. on Machine Learning, Sydney, Australia, 2017, pp. 1263–1272.
Google Scholar
[25]
Z. T. Ying, J. X. You, C. Morris, X. Ren, W. Hamilton, and J. Leskovec, Hierarchical graph representation learning with differentiable pooling, in Proc. Advances in Neural Information Processing Systems, Montréal, Canada, 2018, pp. 4800–4810.
Google Scholar
[26]
H. Y. Xu, R. J. Chen, Y. S. Bai, Z. H. Duan, J. Feng, Y. Z. Sun, and W. Wang, CoSimGNN: Towards large-scale graph similarity computation, arXiv preprint arXiv: 2005.07115, 2020.
Google Scholar
[27]
H. Y. Xu, Z. H. Duan, Y. Y. Wang, J. Feng, R. J. Chen, Q. R. Zhang, and Z. B. Xu, Graph partitioning and graph neural network based hierarchical graph matching for graph similarity computation, Neurocomputing, vol. 439, pp. 348–362, 2021.
DOI Google Scholar
[28]
T. N. Kipf and M. Welling, Semi-supervised classification with graph convolutional networks, arXiv preprint arXiv: 1609.02907, 2016.
Google Scholar
[29]
W. L. Hamilton, R. Ying, and J. Leskovec, Inductive representation learning on large graphs, in Proc. the 31st Int. Conf. on Neural Information Processing Systems, Long Beach, CA, USA, 2017, pp. 1025–1035.
Google Scholar
[30]
J. Chen, T. F. Ma, and C. Xiao, FastGCN: Fast learning with graph convolutional networks via importance sampling, arXiv preprint arXiv: 1801.10247, 2018.
Google Scholar
[31]
K. Y. L. Xu, W. H. Hu, J. Leskovec, and S. Jegelka, How powerful are graph neural networks? arXiv preprint arXiv: 1810.00826, 2018.
Google Scholar
[32]
C. Morris, M. Ritzert, M. Fey, W. L. Hamilton, J. E. Lenssen, G. Rattan, and M. Grohe, Weisfeiler and leman go neural: Higher-order graph neural networks, in Proc. the AAAI Conf. on Artificial Intelligence, Palo Alto, CA, USA, 2019, pp. 4602–4609.
DOI Google Scholar
[33]
P. Velikovi, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, Graph attention networks, arXiv preprint arXiv: 1710.10903, 2017.
Google Scholar
[34]
S. Han, H. B. Dong, X. Y. Teng, X. H. Li, and X. W. Wang, Correlational graph attention-based long short-term memory network for multivariate time series prediction, Appl. Soft Comput., vol. 106, p. 107377, 2021.
DOI Google Scholar
[35]
T. Schreiber, Measuring information transfer, Phys. Rev. Lett., vol. 85, no. 2, pp. 461–464, 2000.
DOI Google Scholar
[36]
D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv: 1412.6980, 2014.
Google Scholar
[37]
L. M. Candanedo, V. Feldheim, and D. Deramaix, Data driven prediction models of energy use of appliances in a low-energy house, Energy Build., vol. 140, pp. 81–97, 2017.
DOI Google Scholar
[38]
Y. Qin, D. J. Song, H. F. Cheng, W. Cheng, G. F. Jiang, and G. W. Cottrell, A dual-stage attention-based recurrent neural network for time series prediction, in Proc. the 26th Int. Joint Conf. on Artificial Intelligence, Melbourne, Australia, 2017, pp. 2627–2633.
DOI Google Scholar
[39]
Y. LeCun and Y. Bengio, Convolutional networks for images, speech, and time series, in The Handbook of Brain Theory and Neural Networks, M. A. Arbib, ed. Cambridge, MA, USA: MIT Press, 1998, pp. 255–258.
[40]
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, Attention is all you need, in Proc. the 31st Int. Conf. on Neural Information Processing Systems, Long Beach, CA, USA, 2017, pp. 6000–6010.
Google Scholar
[41]
J. Z. Cheng, K. Z. Huang, and Z. B. Zheng, Towards better forecasting by fusing near and distant future visions, in Proc. the AAAI Conf. on Artificial Intelligence, Palo Alto, CA, USA, 2020, pp. 3593–3600.
DOI Google Scholar
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 24 April 2021
Revised: 23 September 2021
Accepted: 22 October 2021
Published: 21 July 2022
Issue date: February 2023

Copyright

© The author(s) 2023.

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (No. 62002035) and the Natural Science Foundation of Chongqing (No. cstc2020jcyj-bshX0034).

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return