A power flow convergence adjustment based on deep reinforcement learning with hybrid action space

As the operation modes of power systems become increasingly complex, the difficulty of power flow convergence adjustment also increases. Traditional methods that rely on human expertise suffer from delayed response and low efficiency, making them ill-suited for complex scenarios involving diverse and high-dimensional control variables. To address this challenge, a power flow adjustment method based on deep reinforcement learning with a hybrid action space is proposed. Firstly, a power flow convergence discriminator is developed by integrating physical priors with data-driven techniques to enable real-time identification of whether the power flow converges. The output convergence probability is used as a reward guidance signal in deep reinforcement learning. Then, a reinforcement learning environment for convergence adjustment is defined. The state space integrates system-level statistical features with node-level individual features. The action space encompasses both continuous and discrete control variables. And the reward function combines convergence identification results with feedback from the adjustment process to guide policy optimization toward the feasible region. Next, an Actor-Critic network with a hybrid action space is constructed, which hierarchically decouples and models subtasks including device selection, continuous regulation, and discrete control. Finally, simulation analysis and comparative experiments are conducted on the improved IEEE 39-bus and 118-bus systems. The results demonstrate that the power flow convergence discriminator enhanced with physical priors significantly improves both the accuracy and generalization capability of convergence identification compared to traditional models. Furthermore, the proposed coordinated optimization strategy, which integrates continuous-discrete actions, achieves notable improvements in power flow adjustment efficiency and convergence success rate over existing methods.