Journal Home > Volume 24 , Issue 6

In this paper, we propose a novel Residuals-Based Deep Least Squares Support Vector Machine (RBD-LSSVM). In the RBD-LSSVM, multiple LSSVMs are sequentially connected. The second LSSVM uses the fitting residuals of the first LSSVM as input time series, and the third LSSVM trains the residuals of the second, and so on. The original time series is the input of the first LSSVM. Additionally, to obtain the best hyper-parameters for the RBD-LSSVM, we propose a model validation method based on redundancy test using Omni-Directional Correlation Function (ODCF). This method is based on the fact when a model is appropriate for a given time series, there should be no information or correlation in the residuals. We propose the use of ODCF as a statistic to detect nonlinear correlation between two random variables. Thus, we can select hyper-parameters without encountering overfitting, which cannot be avoided by only cross validation using the validation set. We conducted experiments on two time series: annual sunspot number series and monthly Total Column Ozone (TCO) series in New Delhi. Analysis of the prediction results and comparisons with recent and past studies demonstrate the promising performance of the proposed RBD-LSSVM approach with redundancy test based model selection method for modeling and predicting nonlinear time series.


menu
Abstract
Full text
Outline
About this article

Residuals-Based Deep Least Square Support Vector Machine with Redundancy Test Based Model Selection to Predict Time Series

Show Author's information Yanhua Yu*( )Jie Li
School of Computer, Beijing University of Posts and Telecommunications, Beijing 100876, China.

Abstract

In this paper, we propose a novel Residuals-Based Deep Least Squares Support Vector Machine (RBD-LSSVM). In the RBD-LSSVM, multiple LSSVMs are sequentially connected. The second LSSVM uses the fitting residuals of the first LSSVM as input time series, and the third LSSVM trains the residuals of the second, and so on. The original time series is the input of the first LSSVM. Additionally, to obtain the best hyper-parameters for the RBD-LSSVM, we propose a model validation method based on redundancy test using Omni-Directional Correlation Function (ODCF). This method is based on the fact when a model is appropriate for a given time series, there should be no information or correlation in the residuals. We propose the use of ODCF as a statistic to detect nonlinear correlation between two random variables. Thus, we can select hyper-parameters without encountering overfitting, which cannot be avoided by only cross validation using the validation set. We conducted experiments on two time series: annual sunspot number series and monthly Total Column Ozone (TCO) series in New Delhi. Analysis of the prediction results and comparisons with recent and past studies demonstrate the promising performance of the proposed RBD-LSSVM approach with redundancy test based model selection method for modeling and predicting nonlinear time series.

Keywords: time series prediction, information redundancy, residuals-based deep Least Squares Support Vector Machine (LSSVM), Omni-Directional Correlation Function (ODCF)

References(24)

[1]
Box G. E. P., Jenkins G. M., and Reinsel G. C., Time Series Analysis: Forecasting and Control. Beijing, China: Posts & Telecom Press, 2005.
[2]
Chen H. X., Feng S., Pei X., Zhang Z., and Yao D. Y., Dangerous driving behavior recognition and prevention using an autoregressive time-series model, Tsinghua Science and Technology, vol. 22, no. 6, pp. 682-690, 2017.
[3]
Wang J., Tang L. Y., and Luo Y. Y., A weighted EMD-based prediction model based on TOPSIS and feed forward neural network for noised time series, Knowledge Based System, vol. 132, pp. 167-178, 2017.
[4]
Arash M. and Majid A., Developing a local least-squares support vector machines-based neuro-fuzzy model for nonlinear and chaotic time series prediction, IEEE Transactions on Neural Networks and Learning System, vol. 24, no. 2, pp. 207-218, 2013.
[5]
Han M. and Xu M. L., A hybrid prediction model of multivariate chaotic time series based on error correction, Acta Phys. Sin., vol. 62, no. 12, p. 120510, 2013.
[6]
Guo Z. Q., Wang H. Q., and Liu Q., Financial time seriesforecasting using LPP and SVM optimized by PSO, Soft Computing, vol. 17, no. 5, pp. 805-818, 2013.
[7]
Tang Z. J., Ren F., Peng T., and Wang W. B., A least square support vector machine prediction algorithm for chaotic time series based on the iterative error correction, Acta Phys. Sin., vol. 63, p. 050505, 2014.
[8]
Sapankevych N. I. and Sankar R., Time series prediction using support vector machines: A survey, IEEE Comput. Intell. Mag., vol. 4, no. 2, pp. 24-38, 2009.
[9]
Wang G. L., Yang P. C., and Mao Y. Q., On the application of non-stationary time series prediction based on the SVM method, Acta Phys. Sin., vol. 57, no. 2, pp. 714-719, 2009.
[10]
Yu Y. H. and Song J. D., Redundancy-test-based hyperparameters selection approach for support vector machines to predict time series, Acta Phys. Sin., vol. 61, no. 17, p. 170516, 2012.
[11]
Chen T. T. and Lee S. J., A weighted LS-SVM based learning system for time series forecasting, Information Sciences, vol. 299, pp. 99-116, 2015.
[12]
Zhu Q. M. and Zhang L. F., Development of omni-directional correlation functions for nonlinear model validation, Automatica, vol. 43, no. 9, pp. 1519-1531, 2007.
[13]
Zhang L. F., Zhu Q. M., and Longden A., A correlation-test-based validation procedure for identified neural networks, IEEE Trans. on Neural. Netw., vol. 20, no. 1, pp. 1-13, 2009.
[14]
Vapnik V. N., The Nature of Statistical Learning Theory, 2nd Ed. New York, NY, USA: Springer, 1999.
DOI
[15]
Deng N. Y. and Tian Y. J., Support Vector Machines: Theory, Algorithm and Extension (in Chinese). Beijing, China: Science Press, 2009.
[16]
Suykens J. A. K., Gestel T. V., Brabanter J. D., Moor B. D., and Vandewalle J., Least squares support vector machines, International Journal of Circuit Theory & Applications, vol. 27, no. 6, pp. 605-615, 2002.
DOI
[17]
Ljung L., System Identification: Theory for the User, 2nd Ed. Prentice-Hall Inc., 1999.
DOI
[18]
Mao K. Z. and Billings S. A., Multi-directional model validity tests for non-linear system identification, International Journal of Control, vol. 73, no. 1, p. 32, 2000.
[19]
Brock W. A., Dechert W. D., Scheinkman J., and LeBaron B., A test for independence based on the correlation dimension, Econometric Reviews, vol. 15, pp. 197-235, 1996.
[20]
Zhou Z. H., Ensemble Methods: Foundations and Algorithms. Taylor & Francis, 2012.
DOI
[21]
Yilmaz S. and Oysal Y., Fuzzy wavelet neural network models for prediction and identification of dynamical systems, IEEE Trans. Neural Netw., vol. 21, no. 10, pp. 1599-1609, 2010.
[22]
Cao L. J., Support vector machines experts for time series forecasting, Neurocomputing, vol. 51, pp. 321-339, 2003.
[23]
Weigend A. S., Huberman B. A., and Rumelhart D. E., Predicting the future: A connectionist approach, Int. J. Neural Systems, vol. 1, pp. 193-209, 1990.
[24]
Tong H. and Lim K. S., Threshold autoregressive, limit cycles and cyclical data, Journal of Royal Statistic Society, vol. 42, pp. 245-292, 1980.
Publication history
Copyright
Rights and permissions

Publication history

Received: 10 January 2018
Revised: 22 February 2018
Accepted: 26 February 2018
Published: 05 December 2019
Issue date: December 2019

Copyright

© The author(s) 2019

Rights and permissions

Return