Journal Home > Volume 8 , Issue 3
CSEE Journal of Power and Energy Systems 2022, 8(3): 721-731 https://doi.org/10.17775/CSEEJPES.2020.04520
Open Access | Issue | Published: 30 April 2021

Fault Identification in Power Network Based on Deep Reinforcement Learning

Show Author's Information Mengshi Li,  Huanming Zhang,  Tianyao Ji( ),  Q. H. Wu
School of Electric Power Engineering, South China University of Technology, Guangzhou 510641, China
Keywords:
Artificial intelligence, fault diagnosis, deep reinforcement learning, deep Q network, fault identification, parameter identification, power network
Cite this article:
Li M, Zhang H, Ji T, et al. Fault Identification in Power Network Based on Deep Reinforcement Learning. CSEE Journal of Power and Energy Systems, 2022, 8(3): 721-731. https://doi.org/10.17775/CSEEJPES.2020.04520
Download citation
EndNote(RIS)
BibTeX

471

Views

19

Downloads

Citations

3

Crossref

7

WoS

13

Scopus

0

CSCD

Abstract Full text About this article

With the integration of alternative energy and renewables, the issue of stability and resilience of the power network has received considerable attention. The basic necessity for fault diagnosis and isolation is fault identification and location. The conventional intelligent fault identification method needs supervision, manual labelling of characteristics, and requires large amounts of labelled data. To enhance the ability of intelligent methods and get rid of the dependence on a large amount of labelled data, a novel fault identification method based on deep reinforcement learning (DRL), which has not received enough attention in the field of fault identification, is investigated in this paper. The proposed method uses different faults as parameters of the model to expand the scope of fault identification. In addition, the DRL algorithm can intelligently modify the fault parameters according to the observations obtained from the power network environment, rather than requiring manual and mechanical tuning of parameters. The methodology was tested on the IEEE 14 bus for several scenarios and the performance of the proposed method was compared with that of population-based optimization methods and supervised learning methods. The obtained results have confirmed the feasibility and effectiveness of the proposed method.


menu
Abstract
Full text
Outline
About this article

Fault Identification in Power Network Based on Deep Reinforcement Learning

Show Author's information Mengshi Li, Huanming Zhang, Tianyao Ji( ), Q. H. Wu
School of Electric Power Engineering, South China University of Technology, Guangzhou 510641, China

Abstract

With the integration of alternative energy and renewables, the issue of stability and resilience of the power network has received considerable attention. The basic necessity for fault diagnosis and isolation is fault identification and location. The conventional intelligent fault identification method needs supervision, manual labelling of characteristics, and requires large amounts of labelled data. To enhance the ability of intelligent methods and get rid of the dependence on a large amount of labelled data, a novel fault identification method based on deep reinforcement learning (DRL), which has not received enough attention in the field of fault identification, is investigated in this paper. The proposed method uses different faults as parameters of the model to expand the scope of fault identification. In addition, the DRL algorithm can intelligently modify the fault parameters according to the observations obtained from the power network environment, rather than requiring manual and mechanical tuning of parameters. The methodology was tested on the IEEE 14 bus for several scenarios and the performance of the proposed method was compared with that of population-based optimization methods and supervised learning methods. The obtained results have confirmed the feasibility and effectiveness of the proposed method.

Keywords: Artificial intelligence, fault diagnosis, deep reinforcement learning, deep Q network, fault identification, parameter identification, power network

References(32)

[1]
S. Chatterjee and B. K. S. Roy, “Fast identification of symmetrical or asymmetrical faults during power swings with dual use line relays,” CSEE Journal of Power and Energy Systems, vol. 6, no. 1, pp. 184–192, Mar. 2020, .
DOI Google Scholar
[2]
V. Telukunta, J. Pradhan, A. Agrawal, M. Singh, and S. G. Srivani, “Protection challenges under bulk penetration of renewable energy resources in power systems: a review,” CSEE Journal of Power and Energy Systems, vol. 3, no. 4, pp. 365–379, Dec. 2017, .
DOI Google Scholar
[3]
S. Hasan, “A hybrid circuit breaker based on current commutation approach for multi-feeder DC railway substations,” CSEE Journal of Power and Energy Systems, vol. 5, no. 2, pp. 234–239, Jun. 2019, .
DOI Google Scholar
[4]
J. Ding, X. M. Bai, W. Zhao, Z. Fang, Z. H. Li, and Z. Z. Zhong, “Fault information analysis and diagnosis method of power system based on complex event processing technology,” Proceedings of the CSEE, vol. 27, no. 28, pp. 40–45, Jan. 2007.
Google Scholar
[5]
T. Minakawa, Y. Ichikawa, M. Kunugi, K. Shimada, N. Wada, and M. Utsunomiya, “Development and implementation of a power system fault diagnosis expert system,” IEEE Transactions on Power Systems, vol. 10, no. 2, pp. 932–940, May 1995, .
DOI Google Scholar
[6]
H. C. Shu, Z. Gong, and X. C. Tian, “Fault-section identification for hybrid distribution lines based on principal component analysis and Euclidean distance,” CSEE Journal of Power and Energy Systems, vol. 7, no. 3, pp. 591–603, May 2021, .
DOI Google Scholar
[7]
S. S. Gururajapathy, H. Mokhlis, and H. A. Illias, “Fault location and detection techniques in power distribution systems with distributed generation: a review,” Renewable and Sustainable Energy Reviews, vol. 74, pp. 949–958, Jul. 2017.
DOI Google Scholar
[8]
X. N. Lin, S. H. Ke, Z. T. Li, H. L. Weng, and X. H. Han, “A fault diagnosis method of power systems based on improved objective function and genetic Algorithm-Tabu search,” IEEE Transactions on Power Delivery, vol. 25, no. 3, pp. 1268–1274, Jul. 2010, .
DOI Google Scholar
[9]
Y. L. Wang, S. J. Wen, and D. Y. Wang, “Fuzzy fault diagnosis method based on particle swarm optimization algorithm,” in Proceedings of the 9th International Conference on Fuzzy Systems and Knowledge Discovery, Chongqing, 2012, pp. 307–310, .
DOI
[10]
A. Abel Hafez, W. A. Omran, and Y. G. Hegazy, “A decentralized technique for autonomous service restoration in active radial distribution networks,” IEEE Transactions on Smart Grid, vol. 9, no. 3, pp. 1911–1919, May 2018, .
DOI Google Scholar
[11]
Z. S. Hosseini, M. Mahoor, and A. Khodaei, “AMI-enabled distribution network line outage identification via multi-label SVM,” IEEE Transactions on Smart Grid, vol. 9, no. 5, pp. 5470–5472, Sep. 2018, .
DOI Google Scholar
[12]
F. Dehghani and H. Nezami, “A new fault location technique on radial distribution systems using artificial neural network,” in Proceedings of the 22nd International Conference and Exhibition on Electricity Distribution (CIRED 2013), Stockholm, 2013, pp. 1–4, .
DOI
[13]
S. L. Wen, Y. Wang, Y. Tang, Y. Xu, P. F. Li, and T. Y. Zhao, “Real-time identification of power fluctuations based on LSTM recurrent neural network: a case study on Singapore power system,” IEEE Transactions on Industrial Informatics, vol. 15, no. 9, pp. 5266–5275, Sep. 2019, .
DOI Google Scholar
[14]
M. Fan, Y. L. Liu, X. Zhang, H. Chen, Y. Q. Hu, L. B. Fan, and Q. Yang, “Fault prediction for distribution network based on CNN and LightGBM algorithm,” in Proceedings of the 14th IEEE International Conference on Electronic Measurement & Instruments (ICEMI), Changsha, China, 2019, pp. 1020–1026, .
DOI
[15]
H. T. Shan, Y. Y. Sun, W. J. Zhang, A. Kudreyko, and L. J. Ren, “Reliability analysis of power distribution network based on PSO-DBN,” IEEE Access, vol. 8, pp. 224884–224894, Jul. 2020, .
DOI Google Scholar
[16]
S. Chakraborty, R. Tomsett, R. Raghavendra, D. Harborne, M. Alzantot, F. Cerutti, M. Srivastava, A. Preece, S. Julier, R. M. Rao, T. D. Kelley, D. Braines, M. Sensoy, C. J. Willis, and P. Gurram, “Interpretability of deep learning models: a survey of results,” in Proceedings of 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), San Francisco, CA, 2017, pp. 1–6, .
DOI
[17]
A. L. Yuille and C. X. Liu, “Deep nets: what have they ever done for vision?” International Journal of Computer Vision, vol. 129, no. 3, pp. 781–802, Mar. 2021.
DOI Google Scholar
[18]
Z. D. Zhang, D. X. Zhang, and R. C. Qiu, “Deep reinforcement learning for power system applications: an overview,” CSEE Journal of Power and Energy Systems, vol. 6, no. 1, pp. 213–225, Mar. 2020, .
DOI Google Scholar
[19]
D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton, Y. T. Chen, T. Lillicrap, F. Hui, L. Sifre, G. van den Driessche, T. Graepel, and D. Hassabis, “Mastering the game of go without human knowledge,” Nature, vol. 550, no. 7676, pp. 354–359, Oct. 2017.
DOI Google Scholar
[20]
OpenAI Five. (2018, Oct. 30). OpenAI. [Online]. Available: https://blog.openai.com/openai-five/.
Google Scholar
[21]
N. Golovin and E. Rahm, “Reinforcement learning architecture for Web recommendations,” in Proceedings of 2004 International Conference on Information Technology: Coding and Computing, Las Vegas, NV, USA, 2004, pp. 398–402, .
DOI
[22]
S. X. Gu, E. Holly, T. Lillicrap, and S. Levine, “Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates,” in Proceedings of 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore, 2017, pp. 3389–3396, .
DOI
[23]
R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge: MIT Press, 1998.
DOI
[24]
C. J. C. H. Watkins and P. Dayan, “Q-learning,” Machine Learning, vol. 8, no. 3–4, pp. 279–292, May 1992.
DOI Google Scholar
[25]
Q. H. Huang, R. K. Huang, W. T. Hao, J. Tan, R. Fan, and Z. Y. Huang, “Adaptive power system emergency control using deep reinforcement learning,” IEEE Transactions on Smart Grid, vol. 11, no. 2, pp. 1171–1182, Mar. 2020, .
DOI Google Scholar
[26]
V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis, “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, pp. 529–533, Feb. 2015.
DOI Google Scholar
[27]
V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing Atari with deep reinforcement learning,” arXiv: 1312.5602, 2013.
Google Scholar
[28]
H. Van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with double q-learning,” in Proceedings of the 30th AAAI Conference on Artificial Intelligence, Phoenix, Arizona, USA, 2016, pp. 2094–2100.
DOI
[29]
J. A. Boudreaux, “Design, simulation, and construction of an IEEE 14-bus power system,” M.S. thesis, Louisiana State University, Louisiana, 2018.
[30]
P. K. Lim and D. S. Dorr, “Understanding and resolving voltage sag related problems for sensitive industrial customers,” in Proceedings of 2000 IEEE Power Engineering Society Winter Meeting, Singapore, 2000, pp. 2886–2890, .
DOI
[31]
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, Nov. 1997, .
DOI Google Scholar
[32]
R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Region-based convolutional networks for accurate object detection and segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 1, pp. 142–158, Jan. 2016.
DOI Google Scholar
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 30 August 2020
Revised: 01 November 2020
Accepted: 17 December 2020
Published: 30 April 2021
Issue date: May 2022

Copyright

© 2020 CSEE

Acknowledgements

Acknowledgements

This work is supported by Fundamental Research Funds Program for the Central Universities (No. 2019MS014) and Key-Area Research and Development Program of Guangdong Province (No. 2020B010166004).

Rights and permissions

Return