Journal Home > Volume 2 , Issue 3
Complex System Modeling and Simulation 2022, 2(3): 253-263 https://doi.org/10.23919/CSMS.2022.0013
Open Access | Issue | Published: 30 September 2022

Large-Scale Expensive Optimization with a Switching Strategy

Show Author's Information Mai Sun1,  Chaoli Sun1( ),  Xiaobo Li1,  Guochen Zhang1,  Farooq Akhtar2
School of Computer Science and Technology, Taiyuan University of Science and Technology, Taiyuan 030024, China
Department of Computer Sciences and Information Technology, University of Kotli Azad Jammu and Kashmir, Kotli 11100, Pakistan
Keywords:
large-scale optimization problems, computationally expensive problems, random grouping, surrogate models
Cite this article:
Sun M, Sun C, Li X, et al. Large-Scale Expensive Optimization with a Switching Strategy. Complex System Modeling and Simulation, 2022, 2(3): 253-263. https://doi.org/10.23919/CSMS.2022.0013
Download citation
EndNote(RIS)
BibTeX

518

Views

33

Downloads

Citations

2

Crossref

N/A

WoS

3

Scopus

N/A

CSCD

Abstract Full text About this article

Some optimization problems in scientific research, such as the robustness optimization for the Internet of Things and the neural architecture search, are large-scale in decision space and expensive for objective evaluation. In order to get a good solution in a limited budget for the large-scale expensive optimization, a random grouping strategy is adopted to divide the problem into some low-dimensional sub-problems. A surrogate model is then trained for each sub-problem using different strategies to select training data adaptively. After that, a dynamic infill criterion is proposed corresponding to the models currently used in the surrogate-assisted sub-problem optimization. Furthermore, an escape mechanism is proposed to keep the diversity of the population. The performance of the method is evaluated on CEC’2013 benchmark functions. Experimental results show that the algorithm has better performance in solving expensive large-scale optimization problems.


menu
Abstract
Full text
Outline
About this article

Large-Scale Expensive Optimization with a Switching Strategy

Show Author's information Mai Sun1, Chaoli Sun1( ), Xiaobo Li1, Guochen Zhang1, Farooq Akhtar2
School of Computer Science and Technology, Taiyuan University of Science and Technology, Taiyuan 030024, China
Department of Computer Sciences and Information Technology, University of Kotli Azad Jammu and Kashmir, Kotli 11100, Pakistan

Abstract

Some optimization problems in scientific research, such as the robustness optimization for the Internet of Things and the neural architecture search, are large-scale in decision space and expensive for objective evaluation. In order to get a good solution in a limited budget for the large-scale expensive optimization, a random grouping strategy is adopted to divide the problem into some low-dimensional sub-problems. A surrogate model is then trained for each sub-problem using different strategies to select training data adaptively. After that, a dynamic infill criterion is proposed corresponding to the models currently used in the surrogate-assisted sub-problem optimization. Furthermore, an escape mechanism is proposed to keep the diversity of the population. The performance of the method is evaluated on CEC’2013 benchmark functions. Experimental results show that the algorithm has better performance in solving expensive large-scale optimization problems.

Keywords: large-scale optimization problems, computationally expensive problems, random grouping, surrogate models

References(46)

1
M. Xiao, J. Zhang, K. Cai, X. Cao, and K. Tang, Cooperative co-evolution with weighted random grouping for large-scale crossing waypoints locating in air route network, in Proc. 2011 IEEE 23rd International Conference Tools with Artificial Intelligence, Boca Raton, FL, USA, 2011, pp. 215–222.
2
O. Barrière and E. Lutton, Experimental analysis of a variable size mono-population cooperative-coevolution strategy, in Nature Inspired Cooperative Strategies for Optimization, N. Krasnogor, M. B. Melián-Batista, J. A. M. Pérez, J. M. Moreno-Vega, and D. A. Pelta, eds. Berlin, Germany: Springer, 2008, pp 139–152.
DOI
3
K. Deb and C. Myburgh, Breaking the billion-variable barrier in real-world optimization using a customized evolutionary algorithm, in Proc. Genetic and Evolutionary Computation Conference, Denver, CO, USA, 2016, pp. 653–660.
DOI
4
C. Wang and J. Gao, A new differential evolution algorithm with cooperative coevolutionary selection operator for waveform inversion, in Proc. 2010 IEEE International Geoscience & Remote Sensing Symposium, Honolulu, HI, USA, 2010, pp. 688–690.
DOI
5

K. Deb, A. R. Reddy, and G. Singh, Optimal scheduling of casting sequence using genetic algorithms, Materials and Manufacturing Processes, vol. 18, no. 3, pp. 409–432, 2003.
DOI Google Scholar
6
Y. Liu, X. Yao, Q. Zhao, and T. Higuchi, Scaling up fast evolutionary programming with cooperative coevolution, in Proc. 2001 Congress on Evolutionary Computation, Seoul, Republic of Korea, 2001, pp. 1101–1108.
DOI
7

Z. Yang, K. Tang, and X. Yao, Large scale evolutionary optimization using cooperative coevolution, Information Sciences, vol. 178, no. 15, pp. 2985–2999, 2008.
DOI Google Scholar
8
M. N. Omidvar, Y. Mei, and X. Li, Effective decomposition of large-scale separable continuous functions for cooperative co-evolutionary algorithms, in Proc. 2014 IEEE Congress on Evolutionary Computation (CEC), Beijing, China, 2014, pp. 1305–1312.
DOI
9

M. N. Omidvar, X. Li, Y. Mei, and X. Yao, Cooperative co-evolution with differential grouping for large scale optimization, IEEE Transactions on Evolutionary Computation, vol. 18, no. 3, pp. 378–393, 2013.
DOI Google Scholar
10

M. N. Omidvar, M. Yang, Y. Mei, X. Li, and X. Yao, DG2: A faster and more accurate differential grouping for large-scale black-box optimization, IEEE Transactions on Evolutionary Computation, vol. 21, no. 6, pp. 929–942, 2017.
DOI Google Scholar
11

R. Cheng and Y. Jin, A competitive swarm optimizer for large scale optimization, IEEE Transactions on Cybernetics, vol. 45, no. 2, pp. 191–204, 2014.
DOI Google Scholar
12

R. Cheng and Y. Jin, A social learning particle swarm optimization algorithm for scalable optimization, Information Sciences, vol. 291, pp. 43–60, 2015.
DOI Google Scholar
13

Z. Ren, A. Chen, M. Wang, Y. Yang, Y. Liang, and K. Shang, Bi-hierarchical cooperative coevolution for large scale global optimization, IEEE Access, vol. 8, pp. 41913–41928, 2020.
DOI Google Scholar
14

C. Sun, J. Ding, J. Zeng, and Y. Jin, A fitness approximation assisted competitive swarm optimizer for large scale expensive optimization problems, Memetic Computing, vol. 10, no. 2, pp. 123–134, 2018.
DOI Google Scholar
15

C. Sun, J. Zeng, J. Pan, S. Xue, and Y. Jin, A new fitness estimation strategy for particle swarm optimization, Information Sciences, vol. 221, pp. 355–370, 2013.
DOI Google Scholar
16
I. D. Falco, A. D. Cioppa, and G. A. Trunfio, Large scale optimization of computationally expensive functions: An approach based on parallel cooperative coevolution and fitness metamodeling, in Proc. Genetic and Evolutionary Computation Conference Companion, Berlin, Germany, 2017, pp. 1788–1795.
DOI
17

S. Chen, C. F. N. Cowan, and P. Grant, Orthogonal least squares learning algorithm for radial basis function networks, IEEE Transactions on Neural Networks, vol. 2, no. 2, pp. 302–309, 1991.
DOI Google Scholar
18

Y. Tang, J. Chen, and J. Wei, A surrogate-based particle swarm optimization algorithm for solving optimization problems with expensive black box functions, Engineering Optimization, vol. 45, no. 5, pp. 557–576, 2013.
DOI Google Scholar
19

Y. Jin, A comprehensive survey of fitness approximation in evolutionary computation, Soft Computing, vol. 9, no. 1, pp. 3–12, 2005.
DOI Google Scholar
20

J. Tian, Y. Tan, J. Zeng, C. Sun, and Y. Jin, Multiobjective infill criterion driven Gaussian process-assisted particle swarm optimization of high-dimensional expensive problems, IEEE Transactions on Evolutionary Computation, vol. 23, no. 3, pp. 459–472, 2018.
DOI Google Scholar
21

J. Zhang and A. C. Sanderson, JADE: Adaptive differential evolution with optional external archive, IEEE Transactions on Evolutionary Computation, vol. 13, no. 5, pp. 945–958, 2009.
DOI Google Scholar
22

K. Qiao, J. Liang, B. Qu, K. Yu, C. Yue, and H. Song, Differential evolution with level-based learning mechanism, Complex System Modeling and Simulation, vol. 2, no. 1, pp. 35–58, 2022.
DOI Google Scholar
23

W. Li, X. Ye, Y. Huang, and S. Mahmoodi, Adaptive dimensional learning with a tolerance framework for the differential evolution algorithm, Complex System Modeling and Simulation, vol. 2, no. 1, pp. 59–77, 2022.
DOI Google Scholar
24

Z. Liao and S. Li, Solving nonlinear equations systems with an enhanced reinforcement learning based differential evolution, Complex System Modeling and Simulation, vol. 2, no. 1, pp. 78–95, 2022.
DOI Google Scholar
25

I. D. Falco, A. D. Cioppa, and G. A. Trunfio, Investigating surrogate-assisted cooperative coevolution for large-scale global optimization, Information Sciences, vol. 482, pp. 1–26, 2019.
DOI Google Scholar
26
G. Fu, C. Sun, Y. Tan, G. Zhang, and Y. Jin, A surrogate-assisted evolutionary algorithm with random feature selection for large-scale expensive problems, in Proc. 16th International Conference Parallel Problem Solving from Nature, Leiden, the Netherlands, 2020, pp. 125–139.
DOI
27
S. Liu, H. Wang, W. Peng, and W. Yao, A surrogate-assisted evolutionary feature selection algorithm with parallel random grouping for high-dimensional classification, IEEE Transactions on Evolutionary Computation, doi: 10.1109/TEVC.2022.3149601.
DOI
28

R. Storn and K. Price, Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces, Journal of Global Optimization, vol. 11, no. 4, pp. 341–359, 1997.
DOI Google Scholar
29
M. A. Potter and K. A. D. Jong, A cooperative coevolutionary approach to function optimization, in Proc. Third International Conference on Parallel Problem Solving from Nature, Jerusalem, Israel, 1994, pp. 249–257.
DOI
30

Y. Yao and H. Zhou, The dynamic equilibrium and simulation of mobile internet platform innovation ecosystem: A symbiotic evolution model, Kybernetes, vol. 45, no. 9, pp. 1406–1420, 2016.
DOI Google Scholar
31

Z. Cai and Z. Peng, Cooperative coevolutionary adaptive genetic algorithm in path planning of cooperative multi-mobile robot systems, Journal of Intelligent and Robotic Systems, vol. 33, no. 1, pp. 61–71, 2002.
DOI Google Scholar
32

F. V. D. Bergh and A. P. Engelbrecht, A cooperative approach to particle swarm optimization, IEEE Transactions on Evolutionary Computation, vol. 8, no. 3, pp. 225–239, 2004.
DOI Google Scholar
33

X. Li and X. Yao, Cooperatively coevolving particle swarms for large scale optimization, IEEE Transactions on Evolutionary Computation, vol. 16, no. 2, pp. 210–224, 2011.
DOI Google Scholar
34
Y. -J. Shi, H. -F. Teng, and Z. -Q. Li, Cooperative co-evolutionary differential evolution for function optimization, in Proc. First International Conference on Natural Computation, Changsha, China, 2005, pp. 1080–1088.
DOI
35
Z. Yang, K. Tang, and X. Yao, Differential evolution for high-dimensional function optimization, in Proc. 2007 IEEE Congress on Evolutionary Computation, Singapore, 2007, pp. 3523–3530.
36

Z. -J. Wang, Z. -H. Zhan, S. Kwong, H. Jin, and J. Zhang, Adaptive granularity learning distributed particle swarm optimization for large-scale optimization, IEEE Transactions on Cybernetics, vol. 51, no. 3, pp. 1175–1188, 2020.
DOI Google Scholar
37
G. A. Trunfio, Adaptation in cooperative coevolutionary optimization, in Adaptation and Hybridization in Computational Intelligence, I. Fisrer and I. Fister Jr., eds. Cham, Switzerland: Springer, 2015, pp. 91–109.
DOI
38
Z. Cao, L. Wang, Y. Shi, X. Hei, X. Rong, Q. Jiang, and H. Li, An effective cooperative coevolution framework integrating global and local search for large scale optimization problems, in Proc. 2015 IEEE Congress on Evolutionary Computation, Sendai, Japan, 2015, pp. 1986–1993.
DOI
39

X. Ma, X. Li, Q. Zhang, K. Tang, Z. Liang, W. Xie, and Z. Zhu, A survey on cooperative co-evolutionary algorithms, IEEE Transactions on Evolutionary Computation, vol. 23, no. 3, pp. 421–441, 2018.
DOI Google Scholar
40

Y. Mei, M. N. Omidvar, X. Li, and X. Yao, A competitive divide-and-conquer algorithm for unconstrained large-scale black-box optimization, ACM Transactions on Mathematical Software, vol. 42, no. 2, pp. 1–24, 2016.
DOI Google Scholar
41

B. Liu, Q. Zhang, and G. G. E. Gielen, A Gaussian process surrogate model assisted evolutionary algorithm for medium scale expensive optimization problems, IEEE Transactions on Evolutionary Computation, vol. 18, no. 2, pp. 180–192, 2013.
DOI Google Scholar
42
H. Yu, Y. Tan, J. Zeng, C. Sun, and Y. Jin, Surrogate-assisted hierarchical particle swarm optimization, Information Sciences, vols. 454&455, pp. 59–72, 2018.
DOI
43

H. Wang and Y. Jin, A random forest-assisted evolutionary algorithm for data-driven constrained multiobjective combinatorial optimization of trauma systems, IEEE Transactions on Cybernetics, vol. 50, no. 2, pp. 536–549, 2018.
DOI Google Scholar
44

H. Yu, Y. Tan, C. Sun, and J. Zeng, A generation-based optimal restart strategy for surrogate-assisted social learning particle swarm optimization, Knowledge-Based Systems, vol. 163, pp. 14–25, 2019.
DOI Google Scholar
45
X. Li, K. Tang, M. N. Omidvar, Z. Yang, and K. Qin, Benchmark functions for the CEC 2013 special session and competition on large-scale global optimization, Tech. Rep., Evolutionary Computation and Machine Learning Group, RMIT University, Melbourne, Australia, 2013.
46

C. Sun, Y. Jin, R. Cheng, J. Ding, and J. Zeng, Surrogate-assisted cooperative swarm optimization of high-dimensional expensive problems, IEEE Transactions on Evolutionary Computation, vol. 21, no. 4, pp. 644–660, 2017.
DOI Google Scholar
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 20 June 2022
Revised: 04 July 2022
Accepted: 14 July 2022
Published: 30 September 2022
Issue date: September 2022

Copyright

© The author(s) 2022

Acknowledgements

Acknowledgment

This work was supported in part by the National Natural Science Foundation of China (No. 61876123), Shanxi Key Research and Development Program (No. 202102020101002), and Natural Science Foundation of Shanxi Province (Nos. 201901D111264 and 201901D111262).

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return