Journal Home > Volume 2 , Issue 4

Traditional Belief-Rule-Based (BRB) ensemble learning methods integrate all of the trained sub-BRB systems to obtain better results than a single belief-rule-based system. However, as the number of BRB systems participating in ensemble learning increases, a large amount of redundant sub-BRB systems are generated because of the diminishing difference between subsystems. This drastically decreases the prediction speed and increases the storage requirements for BRB systems. In order to solve these problems, this paper proposes BRBCS-PAES: a selective ensemble learning approach for BRB Classification Systems (BRBCS) based on Pareto-Archived Evolutionary Strategy (PAES) multi-objective optimization. This system employs the improved Bagging algorithm to train the base classifier. For the purpose of increasing the degree of difference in the integration of the base classifier, the training set is constructed by the repeated sampling of data. In the base classifier selection stage, the trained base classifier is binary coded, and the number of base classifiers participating in integration and generalization error of the base classifier is used as the objective function for multi-objective optimization. Finally, the elite retention strategy and the adaptive mesh algorithm are adopted to produce the PAES optimal solution set. Three experimental studies on classification problems are performed to verify the effectiveness of the proposed method. The comparison results demonstrate that the proposed method can effectively reduce the number of base classifiers participating in the integration and improve the accuracy of BRBCS.


menu
Abstract
Full text
Outline
About this article

Selective Ensemble Learning Method for Belief-Rule-Base Classification System Based on PAES

Show Author's information Wanling LiuWeikun WuYingming WangYanggeng Fu( )Yanqing Lin
School of Mathematics and Computer Science, Fuzhou University, Fuzhou 350116, China.
Institute of Decision Sciences, Fuzhou University, Fuzhou 350116, China.

Abstract

Traditional Belief-Rule-Based (BRB) ensemble learning methods integrate all of the trained sub-BRB systems to obtain better results than a single belief-rule-based system. However, as the number of BRB systems participating in ensemble learning increases, a large amount of redundant sub-BRB systems are generated because of the diminishing difference between subsystems. This drastically decreases the prediction speed and increases the storage requirements for BRB systems. In order to solve these problems, this paper proposes BRBCS-PAES: a selective ensemble learning approach for BRB Classification Systems (BRBCS) based on Pareto-Archived Evolutionary Strategy (PAES) multi-objective optimization. This system employs the improved Bagging algorithm to train the base classifier. For the purpose of increasing the degree of difference in the integration of the base classifier, the training set is constructed by the repeated sampling of data. In the base classifier selection stage, the trained base classifier is binary coded, and the number of base classifiers participating in integration and generalization error of the base classifier is used as the objective function for multi-objective optimization. Finally, the elite retention strategy and the adaptive mesh algorithm are adopted to produce the PAES optimal solution set. Three experimental studies on classification problems are performed to verify the effectiveness of the proposed method. The comparison results demonstrate that the proposed method can effectively reduce the number of base classifiers participating in the integration and improve the accuracy of BRBCS.

Keywords: classification, belief-rule-base, pareto-archived evolutionary strategy, selective ensemble

References(24)

[1]
J. B. Yang, J. Liu, J. Wang, H. S Sii, and H. W. Wang, Belief rule-base inference methodology using the evidential reasoning approach-RIMER, IEEE Trans. Syst. Man Cybern. A Syst. Hum., vol. 36, no. 2, pp. 266-285, 2006.
[2]
C. L. Hwang and K. Yoon, Methods for multiple attribute decision making, in Multiple Attribute Decision Making: Methods and Applications A State-of-the-Art Survey, C. L. Hwang and K. Yoon, eds. Springer, 1981, pp. 58-191.
DOI
[3]
A. P. Dempster, A generalization of Bayesian inference, J. Roy. Stat. Soc., vol. 30, no. 2, pp. 205-232, 1968.
[4]
G. Shafer, A Mathematical Theory of Evidence. Princeton, UJ, USA: Princeton University Press, 1976.
DOI
[5]
W. K. Wu, Y. G. Fu, Q. Su, Y. J. Wu, and X. T. Gong, GDA based ensemble learning methods for parameter training in belief rule base, (in Chinese), J. Front. Comput. Sci. Technol., vol. 10, no. 12, pp. 1651-1661, 2016.
[6]
W. K. Wu, L. H. Yang, Y. G. Fu, L. Q. Zhang, and X. T. Gong, Parameter training approach for belief rule base using the accelerating of gradient algorithm, (in Chinese), J. Front. Comput. Sci. Technol., vol. 8, no. 8, pp. 989-1001, 2014.
[7]
J. B. Yang, Rule and utility based evidential reasoning approach for multi-attribute decision analysis under uncertainties, Eur. J. Oper. Res., vol. 131, no. 1, pp. 31-61, 2001.
[8]
W. He, P. L. Qiao, Z. J. Zhou, G. Y Hu, Z. C Feng, and H. Wei, A new belief-rule-based method for fault diagnosis of wireless sensor network, IEEE Access, vol. 6, pp. 9404-9419, 2018.
[9]
Z. J. Zhou, G. Y. Hu, B. C. Zhang, C. H. Hu, Z. G. Zhou, and P. L. Qiao, A model for hidden behavior prediction of complex systems based on belief rule base and power set, IEEE Trans. Syst. Man Cybern. Syst., vol. 48, no. 9, pp. 1649-1655, 2018.
[10]
X. Yin, B. Zhang, Z. Zhou, Z. Wang, and G. Hu, A novel health estimation model for CNC machine tool servo system based on belief-rule-base, in Prognostics and System Health Management Conference, 2017, p. 8079212.
DOI
[11]
Z. J. Zhou, Z. C. Feng, C. H. Hu, F. J. Zhao, Y. M. Zhang, and G. Y. Hu, Fault detection based on belief rule base with online updating attribute weight, in Proc. 32nd Youth Academic Annual Conference of Chinese Association of Automation, Hefei, China, 2017, pp. 272-276.
DOI
[12]
L. L. Chang, Z. J. Zhou, Y. W. Chen, T. J Liao, Y. Hu, and L. H. Yang, Belief rule base structure and parameter joint optimization under disjunctive assumption for nonlinear complex system modeling, IEEE Trans. Syst. Man Cybern. Syst., vol. 48, no. 9, pp. 1542-1554, 2018.
[13]
Y. M. Wang, J. B. Yang, D. L. Xu, and K. S. Chin, The evidential reasoning approach for multiple attribute decision analysis using interval belief degrees, Eur. J. Oper. Res., vol. 175, no. 1, pp. 35-66, 2006.
[14]
F. Luna, A. J. Nebro, and E. Alba, Observations in using grid-enabled technologies for solving multi-objective optimization problems, Parallel Comput., vol. 32, nos. 5&6, pp. 377-393, 2006.
[15]
S. Rostami and A. Shenfield, CMA-PAES: Pareto archived evolution strategy using covariance matrix adaptation for multi-objective optimization, in Proc. 12th UK Workshop on Computational Intelligence, Edinburgh, UK, 2012, pp. 1-8.
DOI
[16]
Q. M. Fan, Multi-objective optimization design of vehicle transmission system based on Pareto optimal theory, in Proc. 2nd Int. Conf. Intelligent Computation Technology and Automation, Changsha, China, 2009, pp. 198-201.
[17]
Y. Y. Jiang, Selective ensemble learning algorithm, in Proc. 2010 Int. Conf. Electrical and Control Engineering, Wuhan, China, 2010, pp. 1859-1862.
[18]
D. W. Corne, J. D. Knowles, and M. J. Oates, The pareto envelope-based selection algorithm for multiobjective optimization, in Proc. 2000 Int. Conf. Parallel Problem Solving from Nature, Paris, France, 2000.
DOI
[19]
R. F. Huang, X. M. Luo, B. Ji, P. Wang, A. Yu, Z. H. Zhai, and J. J. Zhou, Multi-objective optimization of a mixed-flow pump impeller using modified NSGA-II algorithm, Sci. China Technol. Sci., vol. 58, no. 12, pp. 2122-2130, 2015.
[20]
X. H. Wu and Q. Xu, Optimization model of multi-objective distribution based on adaptive grid particle swarm optimization algorithm, (in Chinese), J. Highway Transp. Res. Dev., vol. 27, no. 5, pp. 132-136, 2010.
[21]
Z. H. Zhou, J. X. Wu, and W. Tang, Ensembling neural networks: Many could be better than all, Artif. Intell., vol. 137, nos. 1&2, pp. 239-263, 2002.
[22]
L. L. Chang, Z. J. Zhou, Y. You, L. H. Yang, and Z. G. Zhou, Belief rule based expert system for classification problems with new rule activation and weight calculation procedures, Inform. Sci., vol. 336, pp. 75-91, 2016.
[23]
Q. Q. Ye, L. H. Yang, Y. G. Fu, and X. C. Chen, Classification approach based on improved belief rule-base reasoning, (in Chinese), J. Front. Comput. Sci. Technol., vol. 10, no. 5, pp. 709-721, 2016.
[24]
R. Y. Rubinstein and D. P. Kroese, The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation and Machine Learning. Springer, 2004.
DOI
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 24 March 2019
Revised: 09 April 2019
Accepted: 11 April 2019
Published: 05 August 2019
Issue date: December 2019

Copyright

© The author(s) 2019

Acknowledgements

This research was supported by the National Natural Science Foundation of China (Nos. 71501047 and 61773123) and the Natural Science Foundation of Fujian Province (No. 2019J01647).

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return