A review on deep reinforcement learning for voltage control in new-type power systems

With the increasingly prominent "dual-high" characteristics of the new power system, grid voltage control is confronted with challenges including strong source-load uncertainties, sharp surge of control dimensions, as well as stringent engineering requirements of millisecond-level real-time response. Restricted by reliance on physical models and poor online computational efficiency, traditional analytical methods and heuristic algorithms fail to meet the practical operation demands of power grids. Benefiting from the merits of model-free data-driven modeling, high-dimensional state perception and millisecond-scale online decision-making, deep reinforcement learning (DRL) provides an effective approach to break through the aforementioned bottlenecks. This paper systematically reviews the latest research progress and key technologies of DRL applied to voltage control in new power systems. Firstly, it analyzes the emerging challenges of voltage control in new power systems and the matching advantages of DRL, and briefly introduces its underlying theoretical framework of Markov decision process (MDP). Secondly, on this basis, it thoroughly dissects the solutions and technical limitations of existing studies from four key technical dimensions: collaborative decision-making architecture design, physical security boundary guarantee, adaptive mechanism for operating topologies, and cross-time-scale coordination of heterogeneous regulation resources. Finally, it summarizes the core contradictions existing in current researches such as inherent black-box characteristics and disconnection from idealized simulation environments, and prospects future development directions including constructing high-fidelity verification platforms and provably safe control strategies. This review can offer theoretical references for the engineering application of DRL technology in new power systems.