With the gradual transformation of the global energy structure and the rapid development of renewable energy technologies, microgrid technology has emerged as an important and rapidly growing area in the energy sector. As the core of microgrid operation and control, the energy management system ensures the efficient use of renewable energy and the stable operation of microgrids through precise monitoring and intelligent control. Traditional energy management methods struggle to effectively handle the complex interrelationships among variables in microgrids, whereas deep reinforcement learning (DRL) enables intelligent decision-making by learning through interaction with the environment and adjusting strategies based on feedback signals. To address the high exploration cost and low training efficiency of existing DRL algorithms, this study proposes a microgrid energy management strategy based on a two-stage DRL framework.

The proposed strategy includes two stages: offline and online. First, in the offline stage, linear programming is used to obtain the optimal scheduling results of typical days to construct an expert experience library, and imitation learning is then used to pretrain the agents. This stage involves extracting key information from historical data, such as photovoltaic power, wind power, load demand, and electricity prices, and transforming the data into state–action pairs, thereby forming the pretraining foundation for the agents. Subsequently, in the online stage, the agents interact with the real environment to learn the optimal scheduling strategies for nontypical days. During this stage, having accumulated sufficient knowledge in the offline stage, the agents can significantly improve the environmental tracking accuracy and operational economy. A cliff-walk reward mechanism is introduced to ensure that the agents immediately stop exploring after making decisions that violate constraints, thereby reducing the training cost associated with invalid exploration. Concurrently, the proximal policy optimization (PPO) algorithm is introduced to meet the requirements of continuous action spaces and further improve the performance of the agents.

The proposed algorithm has been validated in a typical microgrid system with three PV stations, one wind turbine, one storage system, and flexible loads. The simulation results show that the convergence speed is significantly improved, and the average daily operating cost is reduced by approximately 14% compared with that of single-stage PPO. A comparison with Double Deep Q-Network, Dueling Deep Q-Network, and Distributed Dueling Deep Q-Network further demonstrates the advantages of the proposed method in achieving optimal performance.

In the two-stage DRL, pretraining in the offline stage enables agents to learn general features and strategies, enabling them to adapt more quickly to new environments and tasks in subsequent missions. Training in the online stage helps agents avoid overfitting to task-specific training data, reducing reliance on such data, lowering the risk of overfitting, and ultimately endowing the strategies with better generalization capability and robustness. Overall, compared with single-stage DRL-based microgrid energy management algorithms, the proposed two-stage DRL-based strategy significantly improves the training efficiency and optimal performance of the agents.

With the rapid integration of information and communication technology into power systems, traditional power grids have evolved into highly interconnected cyber-physical power systems (CPPSs), considerably enhancing observability and controllability. However, this integration exposes CPPSs to severe cybersecurity threats, notably false data injection attacks (FDIAs), which maliciously alter measurement data to mislead operational decisions. Among various forms of FDIAs, load redistribution (LR) attacks are particularly stealthy and dangerous, involving the manipulation of load and line measurements without altering the total load, causing severe operational risks and economic damage. Due to their inherent stealthiness and complexity, LR attacks are often ineffectively detected and inaccurately localized by traditional model-based detection methods. Thus, developing efficient and precise data-driven detection and localization methods is necessary to ensure the reliable operation of CPPSs.

This paper proposes an innovative LR attack detection and localization approach based on a convolutional neural network integrated with a convolutional block attention module (CBAM-CNN). A dual-level optimization model is first constructed to simulate realistic attack scenarios considering different attack intensities and resources. The upper-level model represents attacker strategies aimed at maximizing load shedding using limited resources, while the lower-level model simulates dispatcher responses aiming to minimize load loss. Three distinct LR attack types are defined: manipulation of bus and transmission line measurements and interference with substation data uploads. Comprehensive attack scenarios are simulated on a 38-bus CPPS, generating diverse datasets for model training and validation. The CBAM-CNN approach transforms the attack localization problem into a multi-label classification problem. CNN layers initially extract local features from sparse measurement data, exploiting the inherent network topology and neighborhood relationships within power grids. Subsequently, the CBAM considerably enhances the network’s capability by simultaneously performing channel- and spatial-wise attention to focus on crucial features and suppress irrelevant data, effectively reducing false negatives and improving detection accuracy. Extensive simulations conducted with the 38-bus CPPS model demonstrated the superior performance of the proposed CBAM-CNN method.

Compared with existing methods, including multi-class support vector machine, K-nearest neighbors, extreme gradient boosting, and standard CNN classifiers, the CBAM-CNN consistently achieved higher precision, recall, and F1 scores. Specifically, the proposed method achieved an average precision of 91.63%, a recall of 91.02%, and an F1 score of 91.32%, considerably outperforming other classifiers. Furthermore, detailed scenario analyses indicated that the CBAM-CNN maintained robust performance even with varying attack strengths and resource availability, notably excelling in scenarios with moderate attack intensities and resources, effectively balancing sensitivity and specificity. Radar charts demonstrated that the CBAM-CNN notably improved localization precision across challenging nodes and lines frequently targeted in attacks, highlighting its efficacy in managing sparse and complex attack patterns.

This study successfully developed a highly effective LR attack detection and localization framework combining physical system modeling and advanced data-driven neural network techniques. The dual-level planning model provides realistic, purposeful attack datasets, enhancing training quality and generalizability. The innovative use of CBAM integrated into CNN considerably improved the network's ability to identify and localize stealthy LR attacks accurately. This research underscores the necessity of integrating attention mechanisms in neural networks for cybersecurity applications within CPPSs, offering practical insights and a promising methodology for future security enhancement efforts.