Differential Evolution with Joint Adaptation of Mutation Strategies and Control Parameters via Distributed Proximal Policy Optimization

The mutation operations and related control parameters play important roles in the performance of the differential evolution algorithm. Learning optimal policies for these strategies and parameters through reinforcement learning is a hot topic. However, most of the current studies focus on either mutation strategy selection or the control parameters alone while the others keep fixed or self-adaptive, resulting in deteriorated performances. To address this gap, this paper proposes a framework for the joint adaptation of mutation strategies and related control parameters based on deep reinforcement learning. In this method, the distributed proximal policy optimization algorithm is employed to train the agents to dynamically select the optimal combination of mutation strategies and control parameters. To enhance the agent’s learning of the optimal policy, information derived from fitness landscape analysis is incorporated into the state representations. The training is conducted on the black-box optimization benchmark test problems, which are capable of generating large-scale test instances. Numerical results on the new problems from CEC2013 and CEC2017 test suites, and the real-world application of rover trajectory planning demonstrate that the proposed approach achieves competitive performance compared to state-of-the-art methods. The adaptation behavior and the contribution of learning are also thoroughly analyzed.