Journal Home > Volume 2 , Issue 1
Complex System Modeling and Simulation 2022, 2(1): 78-95 https://doi.org/10.23919/CSMS.2022.0003
Open Access | Issue | Published: 30 March 2022

Solving Nonlinear Equations Systems with an Enhanced Reinforcement Learning Based Differential Evolution

Show Author's Information Zuowen Liao1( ),  Shuijia Li2
Beibu Gulf Ocean Development Research Center, Beibu Gulf University, Qinzhou 535000, China
the School of Computer Science, China University of Geosciences, Wuhan 430074, China
Keywords:
reinforcement learning, multiple roots location, differential evolution, nonlinear equations systems
Cite this article:
Liao Z, Li S. Solving Nonlinear Equations Systems with an Enhanced Reinforcement Learning Based Differential Evolution. Complex System Modeling and Simulation, 2022, 2(1): 78-95. https://doi.org/10.23919/CSMS.2022.0003
Download citation
EndNote(RIS)
BibTeX

1415

Views

1002

Downloads

Citations

20

Crossref

N/A

WoS

17

Scopus

N/A

CSCD

Abstract Full text About this article

Nonlinear equations systems (NESs) arise in a wide range of domains. Solving NESs requires the algorithm to locate multiple roots simultaneously. To deal with NESs efficiently, this study presents an enhanced reinforcement learning based differential evolution with the following major characteristics: (1) the design of state function uses the information on the fitness alternation action; (2) different neighborhood sizes and mutation strategies are combined as optional actions; and (3) the unbalanced assignment method is adopted to change the reward value to select the optimal actions. To evaluate the performance of our approach, 30 NESs test problems and 18 test instances with different features are selected as the test suite. The experimental results indicate that the proposed approach can improve the performance in solving NESs, and outperform several state-of-the-art methods.


menu
Abstract
Full text
Outline
About this article

Solving Nonlinear Equations Systems with an Enhanced Reinforcement Learning Based Differential Evolution

Show Author's information Zuowen Liao1( ), Shuijia Li2
Beibu Gulf Ocean Development Research Center, Beibu Gulf University, Qinzhou 535000, China
the School of Computer Science, China University of Geosciences, Wuhan 430074, China

Abstract

Nonlinear equations systems (NESs) arise in a wide range of domains. Solving NESs requires the algorithm to locate multiple roots simultaneously. To deal with NESs efficiently, this study presents an enhanced reinforcement learning based differential evolution with the following major characteristics: (1) the design of state function uses the information on the fitness alternation action; (2) different neighborhood sizes and mutation strategies are combined as optional actions; and (3) the unbalanced assignment method is adopted to change the reward value to select the optimal actions. To evaluate the performance of our approach, 30 NESs test problems and 18 test instances with different features are selected as the test suite. The experimental results indicate that the proposed approach can improve the performance in solving NESs, and outperform several state-of-the-art methods.

Keywords: reinforcement learning, multiple roots location, differential evolution, nonlinear equations systems

References(40)

1

W. Gong, Z. Liao, X. Mi, L. Wang, and Y. Guo, Nonlinear equations solving with intelligent optimization algorithms: A survey, Complex System Modeling and Simulation, vol. 1, no. 1, pp. 15–32, 2021.
DOI Google Scholar
2

F. Facchinei and C. Kanzow, Generalized Nash equilibrium problems, 4OR, vol. 5, no. 3, pp. 173–210, 2007.
DOI Google Scholar
3

H. D. Chiang and T. Wang, Novel homotopy theory for nonlinear networks and systems and its applications to electrical grids, IEEE Transactions on Control of Network Systems, vol. 5, no. 3, pp. 1051–1060, 2017.
DOI Google Scholar
4

D. Guo, Z. Nie, and L. Yan, The application of noise-tolerant ZD design formula to robots’ kinematic control via time-varying nonlinear equations solving, IEEE Transactions on Systems,Man,and Cybernetics:Systems, vol. 48, no. 12, pp. 2188–2197, 2017.
DOI Google Scholar
5
Y. Luo, A new 3-cycle Newton chaos iteration solution method and its application to mechanism synthesis, in Proc. International Conference on Mechanical Transmissions, Chongqing, China, 2006, pp. 809–812.
6

R. Storn and K. Price, Differential evolution—A simple and efficient heuristic for global optimization over continuous spaces, Journal of Global Optimization, vol. 11, no. 4, pp. 341–359, 1997.
DOI Google Scholar
7

H. Lu, S. Yang, M. Zhao, and S. Cheng, Multi-robot indoor environment map building based on multi-stage optimization method, Complex System Modeling and Simulation, vol. 1, no. 2, pp. 145–161, 2021.
DOI Google Scholar
8

W. Zhang, W. Hou, C. Li, W. Yang, and M. Gen, Multidirection update-based multiobjective particle swarm optimization for mixed no-idle flow-shop scheduling problem, Complex System Modeling and Simulation, vol. 1, no. 3, pp. 176–197, 2021.
DOI Google Scholar
9

W. Gao, G. Li, Q. Zhang, Y. Luo, and Z. Wang, Solving nonlinear equation systems by a two-phase evolutionary algorithm, IEEE Transactions on Systems,Man,and Cybernetics:Systems, vol. 51, no. 9, pp. 5652–5663, 2021.
DOI Google Scholar
10

Z. Liao, W. Gong, X. Yan, L. Wang, and C. Hu, Solving nonlinear equations system with dynamic repulsion-based evolutionary algorithms, IEEE Transactions on Systems,Man,and Cybernetics:Systems, vol. 50, no. 4, pp. 1590–1601, 2020.
DOI Google Scholar
11

W. He, W. Gong, L. Wang, X. Yan, and C. Hu, Fuzzy neighborhood-based differential evolution with orientation for nonlinear equation systems, Knowledge-Based Systems, vol. 182, p. 104796, 2019.
DOI Google Scholar
12

Z. Liao, W. Gong, and L. Wang, A hybrid swarm intelligence with improved ring topology for nonlinear equations, Scientia Sinica Informationis, vol. 50, no. 3, pp. 396–407, 2020.
DOI Google Scholar
13

J. Wu, W. Gong, and L. Wang, A clustering-based differential evolution with different crowding factors for nonlinear equations system, Applied Soft Computing, vol. 98, p. 106733, 2021.
DOI Google Scholar
14

Z. Liao, W. Gong, L. Wang, X. Yan, and C. Hu, A decomposition-based differential evolution with reinitialization for nonlinear equations systems, Knowledge-Based Systems, vol. 191, p. 105312, 2020.
DOI Google Scholar
15

Z. Liao, W. Gong, and L. Wang, Memetic niching-based evolutionary algorithms for solving nonlinear equation system, Expert Systems with Applications, vol. 149, p. 113261, 2020.
DOI Google Scholar
16

V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al., Human-level control through deep reinforcement learning, Nature, vol. 518, no. 7540, pp. 529–533, 2015.
DOI Google Scholar
17

S. Hussein, L. C. Peng, and S. J. Mohamad, A new reinforcement learning-based memetic particle swarm optimizer, Applied Soft Computing, vol. 43, pp. 276–297, 2016.
DOI Google Scholar
18

L. Wang, Z. Pan, and J. Wang, A review of reinforcement learning based intelligent optimization for manufacturing scheduling, Complex System Modeling and Simulation, vol. 1, no. 4, pp. 257–270, 2021.
DOI Google Scholar
19

Z. Hu and W. Gong, Constrained evolutionary optimization based on reinforcement learning using the objective function and constraints, Knowledge-Based Systems, vol. 237, p. 107731, 2021.
DOI Google Scholar
20

Z. Hu, W. Gong, and S. Li, Reinforcement learning-based differential evolution for parameters extraction of photovoltaic models, Energy Reports, vol. 7, pp. 916–928, 2021.
DOI Google Scholar
21

E. Emary, H. Zawbaa, and C. Grosan, Experienced gray wolf optimization through reinforcement learning and neural networks, IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 3, pp. 681–694, 2018.
DOI Google Scholar
22
Y. Liu, H. Lu, S. Cheng, and Y. Shi, An adaptive online parameter control algorithm for particle swarm optimization based on reinforcement learning, in Proc. 2019 IEEE Congress on Evolutionary Computation (CEC), Wellington, New Zealand, 2019, pp. 815–822.https://doi.org/10.1109/CEC.2019.8790035
DOI
23

Z. Li, L. Shi, C. Yue, Z. Shang, and B. Qu, Differential evolution based on reinforcement learning with fitness ranking for solving multimodal multiobjective problems, Swarm and Evolutionary Computation, vol. 49, pp. 234–244, 2019.
DOI Google Scholar
24

M. M. Drugan, Reinforcement learning versus evolutionary computation: A survey on hybrid algorithms, Swarm and Evolutionary Computation, vol. 44, pp. 228–246, 2019.
DOI Google Scholar
25

C. L. Karr, B. Weck, and L. M. Freeman, Solutions to systems of nonlinear equations via a genetic algorithm, Engineering Applications of Artificial Intelligence, vol. 11, no. 3, pp. 369–375, 1998.
DOI Google Scholar
26

W. Song, Y. Wang, H. -X. Li, and Z. Cai, Locating multiple optimal solutions of nonlinear equation systems based on multiobjective optimization, IEEE Transactions on Evolutionary Computation, vol. 19, no. 3, pp. 414–431, 2015.
DOI Google Scholar
27

S. Lü, S. Han, W. Zhou, and J. Zhang, Recruitment-imitation mechanism for evolutionary reinforcement learning, Information Sciences, vol. 553, pp. 172–188, 2021.
DOI Google Scholar
28

M. J. Hirsch, P. M. Pardalos, and M. G. C. Resende, Solving systems of nonlinear equations with continuous GRASP, Nonlinear Analysis:Real World Applications, vol. 10, no. 4, pp. 2000–2006, 2009.
DOI Google Scholar
29

G. C. V. Ramadas and E. M. G. P. Fernandes, Combined mutation differential evolution to solve systems of nonlinear equations, AIP Conference Proceedings, vol. 1558, no. 1, pp. 582–585, 2013.
DOI Google Scholar
30

W. Gong, Y. Wang, Z. Cai, and L. Wang, Finding multiple roots of nonlinear equation systems via a repulsion-based adaptive differential evolution, IEEE Transactions on Systems,Man,and Cybernetics:Systems, vol. 50, no. 4, pp. 1499–1513, 2020.
DOI Google Scholar
31

L. Zhou and C. S. Jiang, New method for solving nonlinear equation systems, Journal of Chinese Computer Systems, vol. 29, no. 9, pp. 1709–1713, 2008.
Google Scholar
32
W. Gao and Y. Li, Solving a new test set of nonlinear equation systems by evolutionary algorithm, IEEE Transactions on Cybernetics, doi: 10.1109/TCYB.2021.3108563.https://doi.org/10.1109/TCYB.2021.3108563
DOI
33

C. Grosan and A. Abraham, A new approach for solving nonlinear equations systems, IEEE Transactions on Systems,Man and Cybernetics—Part A:System and Humans, vol. 38, no. 3, pp. 698–714, 2008.
DOI Google Scholar
34

W. Gong, Y. Wang, Z. Cai, and S. Yang, A weighted biobjective transformation technique for locating multiple optimal solutions of nonlinear equation systems, IEEE Transactions on Evolutionary Computation, vol. 21, no. 5, pp. 697–713, 2017.
DOI Google Scholar
35

W. Gao, Y. Luo, J. Xu, and S. Zhu, Evolutionary algorithm with multiobjective optimization technique for solving nonlinear equation systems, Information Sciences, vol. 541, pp. 345–361, 2020.
DOI Google Scholar
36

A. M. Ibrahim and M. A. Tawhid, A hybridization of differential evolution and monarch butterfly optimization for solving systems of nonlinear equations, Journal of Computational Design and Engineering, vol. 6, no. 3, pp. 354–367, 2019.
DOI Google Scholar
37

B. Y. Qu, P. N. Suganthan, and J. J. Liang, Differential evolution with neighborhood mutation for multimodal optimization, IEEE Transactions on Evolutionary Computation, vol. 16, no. 5, pp. 601–614, 2012.
DOI Google Scholar
38

Z. Wang, Z. Zhan, Y. Lin, W. Yu, and J. Zhang, Automatic niching differential evolution with contour prediction approach for multimodal optimization problems, IEEE Transactions on Evolutionary Computation, vol. 24, no. 1, pp. 114–128, 2020.
DOI Google Scholar
39

H. Zhao, Z. Zhan, Y. Lin, X. Chen, X. Luo, J. Zhang, S. Kwong, and J. Zhang, Local binary pattern-based adaptive differential evolution for multimodal optimization problems, IEEE Transactions on Cybernetics, vol. 50, no. 7, pp. 3343–3357, 2020.
DOI Google Scholar
40

R. Cheng, M. Li, K. Li, and X. Yao, Evolutionary multiobjective optimization-based multimodal optimization: Fitness landscape approximation and peak detection, IEEE Transactions on Evolutionary Computation, vol. 22, no. 5, pp. 692–706, 2018.
DOI Google Scholar
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Published: 30 March 2022
Issue date: March 2022

Copyright

© The author(s) 2022

Acknowledgements

Acknowledgment

This work was partly supported by the Natural Science Foundation of Guangxi Province (No. 2020JJA170038), Special Talent Project of Guangxi Science and Technology Base (No. GuiKe AD21220119), and the High-Level Talents Research Project of Beibu Gulf (No. 2020KYQD06)

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return