Multi-area load frequency cooperative control based on multi-agent deep reinforcement learning

With the integration of large-scale renewable energy, the frequency stability of power systems has been subjected to severe challenges. In this study, an expert-prefilled multi-agent twin delayed deep deterministic policy gradient (EP-MATD3) algorithm based on an expert data pre-filling mechanism is proposed for multi-area load frequency control. Firstly, a multi-area frequency response model including thermal power units, wind turbines, photovoltaic systems, and energy storage systems is first established. Based on the traditional multi-area tie-line power model, coordinated control between regional controllers is incorporated, by which the interconnection between regions is strengthened and unplanned power exchanges are reduced. Then, the multi-agent twin delayed deep deterministic policy gradient (MATD3) algorithm is adopted to mitigate the Q-value overestimation problem inherent in traditional reinforcement learning, and stability and convergence of the control policy are enhanced. Furthermore, within the collaborative control framework of centralized training and decentralized execution, an expert data pre-filling mechanism is introduced during the centralized training stage, whereby the occurrence of invalid actions during random exploration is limited and the convergence of agent training is accelerated. During the decentralized execution stage, unit power outputs are independently adjusted by the trained agents according to the real-time system states of their respective regions, enabling effective suppression of frequency fluctuations. Through simulation on a three-area power system, it is demonstrated that, compared with traditional methods, the proposed EP-MATD3 control strategy achieves a significant reduction in training time and effectively decreases system frequency deviations under continuous step-load and photovoltaic fluctuation disturbances, thereby verifying its effectiveness and superiority in the frequency control of complex power systems.