Journal Home > Volume 29 , Issue 4

In order to support the perception and defense of the operation risk of the medium and low voltage distribution system, it is crucial to conduct data mining on the time series generated by the system to learn anomalous patterns, and carry out accurate and timely anomaly detection for timely discovery of anomalous conditions and early alerting. And edge computing has been widely used in the processing of Internet of Things (IoT) data. The key challenge of univariate time series anomaly detection is how to model complex nonlinear time dependence. However, most of the previous works only model the short-term time dependence, without considering the periodic long-term time dependence. Therefore, we propose a new Hierarchical Attention Network (HAN), which introduces seven day-level attention networks to capture fine-grained short-term time dependence, and uses a week-level attention network to model the periodic long-term time dependence. Then we combine the day-level feature learned by day-level attention network and week-level feature learned by week-level attention network to obtain the high-level time feature, according to which we can calculate the anomaly probability and further detect the anomaly. Extensive experiments on a public anomaly detection dataset, and deployment in a real-world medium and low voltage distribution system show the superiority of our proposed framework over state-of-the-arts.


menu
Abstract
Full text
Outline
About this article

Univariate Time Series Anomaly Detection Based on Hierarchical Attention Network

Show Author's information Zexi Chen1Dongqiang Jia1( )Yushu Sun2Lin Yang1Wenjie Jin1Ruoxi Liu1
State Grid Beijing Electric Power Company, Beijing 100031, China
Institute of Electrical Engineering, Chinese Academy of Sciences, Beijing 100190, China

Abstract

In order to support the perception and defense of the operation risk of the medium and low voltage distribution system, it is crucial to conduct data mining on the time series generated by the system to learn anomalous patterns, and carry out accurate and timely anomaly detection for timely discovery of anomalous conditions and early alerting. And edge computing has been widely used in the processing of Internet of Things (IoT) data. The key challenge of univariate time series anomaly detection is how to model complex nonlinear time dependence. However, most of the previous works only model the short-term time dependence, without considering the periodic long-term time dependence. Therefore, we propose a new Hierarchical Attention Network (HAN), which introduces seven day-level attention networks to capture fine-grained short-term time dependence, and uses a week-level attention network to model the periodic long-term time dependence. Then we combine the day-level feature learned by day-level attention network and week-level feature learned by week-level attention network to obtain the high-level time feature, according to which we can calculate the anomaly probability and further detect the anomaly. Extensive experiments on a public anomaly detection dataset, and deployment in a real-world medium and low voltage distribution system show the superiority of our proposed framework over state-of-the-arts.

Keywords: edge computing, anomaly detection, univariate time series, self-attention

References(45)

[1]

R. A. Leon, V. Vittal, and G. Manimaran, Application of sensor network for secure electric energy infrastructure, IEEE Trans. Power Deliv., vol. 22, no. 2, pp. 1021–1028, 2007.

[2]
N. Laptev, S. Amizadeh, and I. Flint, Generic and scalable framework for automated time-series anomaly detection, in Proc. 21th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Sydney, Australia, 2015, pp. 1939–1947.
DOI
[3]
Z. Li, X. Xu, T. Hang, H. Xiang, Y. Cui, L. Qi, and X. Zhou, A knowledge-driven anomaly detection framework for social production system, IEEE Trans. Comput. Soc. Syst. doi: 10.1109/TCSS.2022.3217790.
DOI
[4]

Y. Jia, B. Liu, W. Dou, X. Xu, X. Zhou, L. Qi, and Z. Yan, CroApp: A CNN-based resource optimization approach in edge computing environment, IEEE Trans. Ind. Inform., vol. 18, no. 9, pp. 6300–6307, 2022.

[5]

S. Chen, Y. Tao, D. Yu, F. Li, and B. Gong, Distributed learning dynamics of multi-armed bandits for edge intelligence, J. Syst. Archit., vol. 114, p. 101919, 2021.

[6]

S. Chen, Y. Tao, D. Yu, F. Li, B. Gong, and X. Cheng, Privacy-preserving collaborative learning for multiarmed bandits in IoT, IEEE Internet Things J., vol. 8, no. 5, pp. 3276–3286, 2021.

[7]

H. Yu, Z. Chen, X. Zhang, X. Chen, F. Zhuang, H. Xiong, and X. Cheng, FedHAR: Semi-supervised online learning for personalized federated human activity recognition, IEEE Trans. Mob. Comput., vol. 22, no. 6, pp. 3318–3332, 2023.

[8]

V. Zeufack, D. Kim, D. Seo, and A. Lee, An unsupervised anomaly detection framework for detecting anomalies in real time through network system’s log files analysis, High Confid. Comput., vol. 1, no. 2, p. 100030, 2021.

[9]

J. Chen, H. Huang, and H. Chen, Informer: Irregular traffic detection for containerized microservices RPC in the real world, High Confid. Comput., vol. 2, no. 2, p. 100050, 2022.

[10]

S. Wu, S. Shen, X. Xu, Y. Chen, X. Zhou, D. Liu, X. Xue, and L. Qi, Popularity-aware and diverse web APIs recommendation based on correlation graph, IEEE Trans. Comput. Soc. Syst., vol. 10, no. 2, pp. 771–782, 2023.

[11]

X. Zhou, X. Xu, W. Liang, Z. Zeng, and Z. Yan, Deep-learning-enhanced multitarget detection for end-edge-cloud surveillance in smart IoT, IEEE Internet Things J., vol. 8, no. 16, pp. 12588–12596, 2021.

[12]

L. Qi, Y. Yang, X. Zhou, W. Rafique, and J. Ma, Fast anomaly identification based on multiaspect data streams for intelligent intrusion detection toward secure industry 4.0, IEEE Trans. Ind. Inform., vol. 18, no. 9, pp. 6503–6511, 2022.

[13]
H. Z. Moayedi and M. A. Masnadi-Shirazi, ARIMA model for network traffic prediction and anomaly detection, in Proc. 2008 Int. Symp. Information Technology, Kuala Lumpur, Malaysia, 2008, pp. 1–6.
DOI
[14]
F. Knorn and D. J. Leith, Adaptive Kalman filtering for anomaly detection in software appliances, in Proc. IEEE INFOCOM Workshops, Phoenix, AZ, USA, 2008, pp. 1–6.
DOI
[15]
M. Amer, M. Goldstein, and S. Abdennadher, Enhancing one-class support vector machines for unsupervised anomaly detection, in Proc. ACM SIGKDD Workshop on Outlier Detection and Description, Chicago, IL, USA, 2013, pp. 8–15.
DOI
[16]

S. M. Erfani, S. Rajasegarar, S. Karunasekera, and C. Leckie, High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning, Pattern Recognit., vol. 58, pp. 121–134, 2016.

[17]

Z. Ding and M. Fei, An anomaly detection approach based on isolation forest algorithm for streaming data using sliding window, IFAC Proc. Vol., vol. 46, no. 20, pp. 12–17, 2013.

[18]
F. Angiulli and C. Pizzuti, Fast outlier detection in high dimensional spaces, in Proc. 6th European Conf. Principles of Data Mining and Knowledge Discovery, 2002, Helsinki, Finland, 2002, pp. 15–27.
DOI
[19]
R. Zhang, S. Dong, X. Nie, and S. Xiao, Feedforward neural network for time series anomaly detection, arXiv preprint arXiv: 1812.08389, 2018.
[20]

S. Hochreiter and J. Schmidhuber, Long short-term memory, Neural Comput., vol. 9, no. 8, pp. 1735–1780, 1997.

[21]

H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, Informer: Beyond efficient transformer for long sequence time-series forecasting, Proc. AAAI Conf. Artif. Intell., vol. 35, no. 12, pp. 11106–11115, 2021.

[22]
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, Attention is all you need, in Proc. 31st Int. Conf. on Neural Information Processing Systems, Long Beach, CA, USA, 2017, pp. 6000–6010.
[23]

Z. Chen, D. Chen, X. Zhang, Z. Yuan, and X. Cheng, Learning graph structures with transformer for multivariate time-series anomaly detection in IoT, IEEE Internet Things J., vol. 9, no. 12, pp. 9179–9189, 2022.

[24]
P. Malhotra, L. Vig, G. M. Shroff, and P. Agarwal, Long short term memory networks for anomaly detection in time series, in Proc. 23rd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Bruges, Belgium, 2015, pp. 89–94.
[25]

M. Munir, S. A. Siddiqui, A. Dengel, and S. Ahmed, DeepAnT: A deep learning approach for unsupervised anomaly detection in time series, IEEE Access, vol. 7, pp. 1991–2005, 2018.

[26]
C. C. Aggarwal, Time series and multidimensional streaming outlier detection, in Outlier Analysis, C. C. Aggarwal, ed. Cham, Switzerland: Springer, 2017, pp. 273–310.
DOI
[27]
H. Xu, W. Chen, N. Zhao, Z. Li, J. Bu, Z. Li, Y. Liu, Y. Zhao, D. Pei, Y. Feng, et al., Unsupervised anomaly detection via variational auto-encoder for seasonal KPIs in web applications, in Proc. 2018 World Wide Web Conf., Lyon, France, 2018, pp. 187–196.
DOI
[28]

D. Park, Y. Hoshi, and C. C. Kemp, A multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder, IEEE Robot. Autom. Lett., vol. 3, no. 3, pp. 1544–1551, 2018.

[29]
D. Li, D. Chen, B. Jin, L. Shi, J. Goh, and S. K. Ng, MAD-GAN: Multivariate anomaly detection for time series data with generative adversarial networks, in Proc. 28th Int. Conf. Artificial Neural Networks, Munich, Germany, 2019, pp. 703–716.
DOI
[30]
J. Audibert, P. Michiardi, F. Guyard, S. Marti, and M. A. Zuluaga, USAD: Unsupervised anomaly detection on multivariate time series, in Proc. 26th ACM SIGKDD Int. Conf. Knowledge Discovery & Data Mining, Virtual Event, CA, USA, 2020, pp. 3395–3404.
[31]

Y. Dwivedi and S. S. Rao, A test for second-order stationarity of a time series based on the discrete Fourier transform, J. Time Ser. Anal., vol. 32, no. 1, pp. 68–91, 2011.

[32]

L. Breiman, Random forests, Mach. Learn., vol. 45, no. 1, pp. 5–32, 2001.

[33]

M. A. Hearst, S. T. Dumais, E. Osuna, J. Platt, and B. Scholkopf, Support vector machines, IEEE Intell. Syst. Appl., vol. 13, no. 4, pp. 18–28, 1998.

[34]
D. Liu, Y. Zhao, H. Xu, Y. Sun, D. Pei, J. Luo, X. Jing, and M. Feng, Opprentice: Towards practical and automatic anomaly detection through machine learning, in Proc. 2015 Internet Measurement Conference, Tokyo, Japan, 2015, pp. 211–224.
DOI
[35]
W. Qiu, Y. Wu, G. Wang, S. X. Yang, J. Bai, and J. Li, A novel unsupervised anomaly detection based on robust principal component classifier, in Proc. Defense and Security Symposium, Orlando (Kissimmee), FL, USA, 2006, pp. 260–268.
DOI
[36]

S. Chen, Y. Wang, D. Yu, J. Ren, C. Xu, and Y. Zheng, Privacy-enhanced decentralized federated learning at dynamic edge, IEEE Trans. Comput., vol. 72, no. 8, pp. 2165–2180, 2023.

[37]
Y. Yuan, J. Yu, X. Cheng, Z. Zou, D. Yu, and Z. Cai, Decentralized parallel SGD based on weight-balancing for intelligent IoV, IEEE Trans. Intell. Transp. Syst. doi: 10.1109/TITS.2022.3216709.
DOI
[38]

Z. Xie, Y. Huang, D. Yu, R. M. Parizi, Y. Zheng, and J. Pang, FedEE: A federated graph learning solution for extended enterprise collaboration, IEEE Trans. Ind. Inform., vol. 19, no. 7, pp. 8061–8071, 2023.

[39]

S. Chen, D. Yu, Y. Zou, J. Yu, and X. Cheng, Decentralized wireless federated learning with differential privacy, IEEE Trans. Ind. Inform., vol. 18, no. 9, pp. 6273–6282, 2022.

[40]
S. Chen, D. Yu, F. Li, Z. Zou, W. Liang, and X. Cheng, PPAR: A privacy-preserving adaptive ranking algorithm for multi-armed-bandit crowdsourcing, in Proc. 2022 IEEE/ACM 30th Int. Symp. Quality of Service (IWQoS), Oslo, Norway, 2022, pp. 1–10.
DOI
[41]

D. Yu, Z. Zou, S. Chen, Y. Tao, B. Tian, W. Lv, and X. Cheng, Decentralized parallel SGD with privacy preservation in vehicular networks, IEEE Trans. Veh. Technol., vol. 70, no. 6, pp. 5211–5220, 2021.

[42]

Y. Liu, D. Li, S. Wan, F. Wang, W. Dou, X. Xu, S. Li, R. Ma, and L. Qi, A long short-term memory-based model for greenhouse climate prediction, Int. J. Intell. Syst., vol. 37, no. 1, pp. 135–151, 2022.

[43]

Y. Liu, H. Wu, K. Rezaee, M. R. Khosravi, O. I. Khalaf, A. A. Khan, D. Ramesh, and L. Qi, Interaction-enhanced and time-aware graph convolutional network for successive point-of-interest recommendation in traveling enterprises, IEEE Trans. Ind. Inform., vol. 19, no. 1, pp. 635–643, 2023.

[44]

L. Qi, Y. Liu, Y. Zhang, X. Xu, M. Bilal, and H. Song, Privacy-aware point-of-interest category recommendation in Internet of Things, IEEE Internet Things J., vol. 9, no. 21, pp. 21398–21408, 2022.

[45]
A. Ihler, J. Hutchins, and P. Smyth, Adaptive event detection with time-varying Poisson processes, in Proc. 12th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Philadelphia, PA, USA, 2006, pp. 207–216.
DOI
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 01 June 2023
Revised: 19 June 2023
Accepted: 13 July 2023
Published: 09 February 2024
Issue date: August 2024

Copyright

© The Author(s) 2024.

Acknowledgements

Acknowledgment

This work was supported by the Science and Technology Project named “Research on Risk Perception and Defense System for Medium and Low Voltage Distribution System Operation Based on Data Mining” of State Grid Beijing Electric Power Company (No. 520202220002).

Rights and permissions

The articles published in this open access journal are distributed under the terms of theCreative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return