Coordinated Optimal Dispatching Approach for Integrated Energy System Cluster Based on Improved Deep Reinforcement Learning

Integrated energy system (IES) is becoming a mainstream form of energy system decarbonization benefiting from efficient use of multi-energy carriers and user-friendliness to distributed energy resources. Along with development of IES, coordinated operation is necessary to further improve general efficiency of the IES cluster. Conventional distributed optimization methods are difficult to coordinate to optimize IES cluster operation, considering the balance of computational time, convergence performance, and initial value determination. This paper proposes a coordinated optimal dispatching approach for the IES cluster based on improved deep reinforcement learning. First, a distributed scheduling framework of the IES cluster with a cluster energy coordination center (CECC) as the upper coordinator and several area energy management centers (AEMCs) as the lower managers is introduced. Then, the energy interaction model and pricing strategy in the IES cluster that can facilitate multi-energy transactions are established. Finally, an improved multi-agent deep deterministic policy gradient (MADDPG) algorithm with a compression mechanism of initial state space is proposed for the model solution. Results show that compared with the conventional distributed algorithm of the alternating direction method of multipliers (ADMM), the improved MADDPG increases solving speed by 40 times while guaranteeing convergence and optimality of the solution. Besides, after considering energy transactions among IESs in the cluster, the total operating cost is reduced by 9.74%, and the self-sufficient ratio is raised to 0.92, reflecting the necessity of energy complementarity.