Journal Home > Volume 26 , Issue 6
Tsinghua Science and Technology 2021, 26(6): 894-905 https://doi.org/10.26599/TST.2020.9010050
Open Access | Issue | Published: 09 June 2021

Security Issues and Defensive Approaches in Deep Learning Frameworks

Show Author's Information Hongsong Chen( ),  Yongpeng Zhang,  Yongrui Cao,  Jing Xie
Department of Computer Science, University of Science and Technology Beijing (USTB), Beijing 100083, China
Beijing Key Laboratory of Knowledge Engineering for Materials Science, Beijing 100083, China
Department of Computer Science and Technology, University of Science and Technology Beijing (USTB), Beijing 100083, China
Defense Electronics Institute, China Industrial Control System Cyber Emergency Response Team, Beijing 100040, China
Keywords:
adversarial examples, deep learning frameworks, defensive approaches, security issues
Cite this article:
Chen H, Zhang Y, Cao Y, et al. Security Issues and Defensive Approaches in Deep Learning Frameworks. Tsinghua Science and Technology, 2021, 26(6): 894-905. https://doi.org/10.26599/TST.2020.9010050
Download citation
EndNote(RIS)
BibTeX

757

Views

57

Downloads

Citations

20

Crossref

17

WoS

20

Scopus

3

CSCD

Abstract Full text About this article

Deep learning frameworks promote the development of artificial intelligence and demonstrate considerable potential in numerous applications. However, the security issues of deep learning frameworks are among the main risks preventing the wide application of it. Attacks on deep learning frameworks by malicious internal or external attackers would exert substantial effects on society and life. We start with a description of the framework of deep learning algorithms and a detailed analysis of attacks and vulnerabilities in them. We propose a highly comprehensive classification approach for security issues and defensive approaches in deep learning frameworks and connect different attacks to corresponding defensive approaches. Moreover, we analyze a case of the physical-world use of deep learning security issues. In addition, we discuss future directions and open issues in deep learning frameworks. We hope that our research will inspire future developments and draw attention from academic and industrial domains to the security of deep learning frameworks.


menu
Abstract
Full text
Outline
About this article

Security Issues and Defensive Approaches in Deep Learning Frameworks

Show Author's information Hongsong Chen( ), Yongpeng Zhang, Yongrui Cao, Jing Xie
Department of Computer Science, University of Science and Technology Beijing (USTB), Beijing 100083, China
Beijing Key Laboratory of Knowledge Engineering for Materials Science, Beijing 100083, China
Department of Computer Science and Technology, University of Science and Technology Beijing (USTB), Beijing 100083, China
Defense Electronics Institute, China Industrial Control System Cyber Emergency Response Team, Beijing 100040, China

Abstract

Deep learning frameworks promote the development of artificial intelligence and demonstrate considerable potential in numerous applications. However, the security issues of deep learning frameworks are among the main risks preventing the wide application of it. Attacks on deep learning frameworks by malicious internal or external attackers would exert substantial effects on society and life. We start with a description of the framework of deep learning algorithms and a detailed analysis of attacks and vulnerabilities in them. We propose a highly comprehensive classification approach for security issues and defensive approaches in deep learning frameworks and connect different attacks to corresponding defensive approaches. Moreover, we analyze a case of the physical-world use of deep learning security issues. In addition, we discuss future directions and open issues in deep learning frameworks. We hope that our research will inspire future developments and draw attention from academic and industrial domains to the security of deep learning frameworks.

Keywords: adversarial examples, deep learning frameworks, defensive approaches, security issues

References(38)

[1]
W. W. Jiang and L. Zhang, Geospatial data to images: A deep-learning framework for traffic forecasting, Tsinghua Science and Technology, vol. 24, no. 1, pp. 52-64, 2019.
DOI Google Scholar
[2]
L. Zhang, C. B. Xu, Y. H. Gao, Y. Han, X. J. Du, and Z. H. Tian, Improved Dota2 lineup recommendation model based on a bidirectional LSTM,Tsinghua Science and Technology, vol. 25, no. 6, pp. 712-720, 2020.
DOI Google Scholar
[3]
H. M. Huang, J. H. Lin, L. Y. Wu, B. Fang, Z. K. Wen, and F. C. Sun, Machine learning-based multi-modal information perception for soft robotic hands, Tsinghua Science and Technology, vol. 25, no. 2, pp. 255-269, 2020.
DOI Google Scholar
[4]
X. Y. Yuan, P. He, Q. L. Zhu, and X. L. Li, Adversarial examples: Attacks and defenses for deep learning, IEEE Trans. Neural Netw. Learn. Syst., vol. 30, no. 9, pp. 2805-2824, 2019.
DOI Google Scholar
[5]
J. C. Hu, J. F. Chen, L. Zhang, Y. S. Liu, Q. H. Bao, H. Ackah-Arthur, and C. Zhang, A memory-related vulnerability detection approach based on vulnerability features, Tsinghua Science and Technology, vol. 25, no. 5, pp. 604-613, 2020.
DOI Google Scholar
[6]
C. Szegedy, W. Zaremba, I. Sutskever I, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, Intriguing properties of neural networks, arXiv preprint arXiv: 1312.6199, 2013.
Google Scholar
[7]
A. Athalye, N. Carlini, and D. Wagner, Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples, arXiv preprint arXiv: 1802.00420, 2018.
Google Scholar
[8]
Y. T. Xiao, C. M. Pun, and J. Z. Zhou, Generating adversarial perturbation with root mean square gradient, arXiv preprint arXiv: 1901.03706, 2019.
Google Scholar
[9]
I. J. Goodfellow, J. Shlens, and C. Szegedy, Explaining and harnessing adversarial examples, arXiv preprint arXiv: 1412.6572, 2014.
Google Scholar
[10]
J. W. Su, D. V. Vargas, and K. Sakurai, One pixel attack for fooling deep neural networks, IEEE Trans. Evol. Comput., vol. 23, no. 5, pp. 828-841, 2019.
DOI Google Scholar
[11]
W. He, J. Wei, X. Y. Chen, N. Carlini, and D. Song, Adversarial example defense: Ensembles of weak defenses are not strong, in Proc 11th USENIX Workshop on Offensive Technologies, Vancouver, Canada, 2017.
[12]
G. W. Xu, H. W. Li, H. Ren, K. Yang, and R. H. Deng, Data security issues in deep learning: Attacks, countermeasures, and opportunities, IEEE Comm. Mag., vol. 57, no. 11, pp. 116-122, 2019.
DOI Google Scholar
[13]
M. I. Tariq, N. A. Memon, S. Ahmed, S. Tayyaba, M. T. Mushtaq, N. A. Mian, M. Imran, and M. W. Ashraf, A review of deep learning security and privacy defensive techniques, Mobile Inf. Syst., vol. 2020, p. 6535834, 2020.
DOI Google Scholar
[14]
H. Bae, J. Jang, D. Jung, H. Jang, H. Ha, and S. Yoon, Security and privacy issues in deep learning, arXiv preprint arXiv: 1807.11655, 2018.
Google Scholar
[15]
S. L. Qiu, Q. H. Liu, S. J. Zhou, and C. J. Wu, Review of artificial intelligence adversarial attack and defense technologies. Appl. Sci., vol. 9, no. 5,p. 909.
DOI Google Scholar
[16]
S. M. Moosavi-Dezfooli, A. Fawzi and P. Frossard, DeepFool: A simple and accurate method to fool deep neural networks, presented at 2016 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 2574-2582.
DOI
[17]
Q. X. Xiao, K. Li, D. Y. Zhang, and W. L. Xu, Security risks in deep learning implementations, presented at 2018 IEEE Security and Privacy Workshops (SPW), San Francisco, CA, USA, 2018, pp. 123-128.
DOI
[18]
N. Papernot, P. McDaniel, S. Jha, M. Fredrikson, Z. B. Celik, and A. Swami, The limitations of deep learning in adversarial settings, presented at 2016 IEEE European Symp. Security and Privacy (EuroS&P), Saarbrucken, Germany, 2016, pp. 372-387.
DOI
[19]
A. Kurakin, I. Goodfellow, and S. Bengio, Adversarial examples in the physical world, arXiv preprint arXiv: 1607.02533, 2016.
Google Scholar
[20]
S. Huang, N. Papernot, I. Goodfellow, Y. Duan, and P. Abbeel,Adversarial attacks on neural network policies, arXiv preprint arXiv: 1702.02284, 2017.
Google Scholar
[21]
H. W. Zhang, Y. Avrithis, T. Furon, and L. Amsaleg, Walking on the edge: Fast, low-distortion adversarial examples, arXiv preprint arXiv: 1912.02153, 2019.
Google Scholar
[22]
B. Nelson, B. I. P. Rubinstein, L. Huang, A. D. Joseph, S. J. Lee, S. Rao, and J. D. Tygar, Query strategies for evading convex-inducing classifiers, J. Mach. Learn. Res., vol. 13, pp. 1293-1332, 2012.
Google Scholar
[23]
G. Ateniese, G. Felici, L. V. Mancini, A. Spognardi, A. Villani, and D. Vitali, Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers, arXiv preprint arXiv: 1306.4447, 2013.
Google Scholar
[24]
N. Narodytska and S. P. Kasiviswanathan, Simple black-box adversarial perturbations for deep networks, arXiv preprint arXiv: 1612.06299, 2016.
Google Scholar
[25]
P. Y. Chen, H. Zhang, Y. Sharma, J. F. Yi, and C. J. Hsieh, ZOO: Zerothorder optimization based black-box attacks to deep neural networks without training substitute models, in Proc. 10th ACM Workshop on Artificial Intelligence and Security, Dallas, TX, USA, 2017, pp. 15-26.
DOI
[26]
H. S. Ye, Z. C. Huang, C. Fang, C. J. Li, and T. Zhang, Hessian-aware zeroth-order optimization for black-box adversarial attack, arXiv preprint arXiv: 1812.11377, 2018.
Google Scholar
[27]
X. R. Li, S. L. Ji, M. Han, J. T. Ji, Z. Y. Ren, Y. S. Liu, and C. M. Wu, Adversarial examples versus cloud-based detectors: A black-box empirical study, arXiv preprint arXiv: 1901.01223, 2019.
Google Scholar
[28]
S. Saxena, TextDecepter: Hard label black box attack on text classifiers, arXiv preprint arXiv: 2008.06860, 2020.
Google Scholar
[29]
A. Zimba, H. S. Chen, and Z. S. Wang, Bayesian network based weighted APT attack paths modeling in cloud computing, Future Generation Comput. Syst., vol. 96, pp. 525-537,2019.
DOI Google Scholar
[30]
H. S. Chen, C. X. Meng, Z. G. Shan, Z. C. Fu, and B. K. Bhargava, A novel low-rate denial of service attack detection approach in zigbee wireless sensor network by combining Hilbert-Huang transformation and trust evaluation, IEEE Access, vol. 7, pp. 32 853-32 866, 2019.
DOI Google Scholar
[31]
J. Steinhardt, P. W. Koh, and P. Liang, Certified defenses for data poisoning attacks, presented at 31st Conf. Neural Information Proc. Systems, Long Beach, CA, USA, 2017, pp. 3517-3529.
[32]
P. W. Koh and P. Liang, Understanding black-box predictions via influence functions, arXiv preprint arXiv: 1703.04730, 2017.
Google Scholar
[33]
A. Paudice, L. Muñoz-González, A. Gyorgy, and E. C. Lupu, Detection of adversarial training examples in poisoning attacks through anomaly detection, arXiv preprint arXiv: 1802.03041, 2018.
Google Scholar
[34]
A. Paudice, L. Muñoz-González, and E. C. Lupu, Label sanitization against label flipping poisoning attacks, in Joint European Conf. Machine Learning and Knowledge Discovery in Databases, A. Paudice and L. Muñoz-González, eds. Cham, Germany: Springer, 2018, pp. 5-15.
[35]
N. Carlini and D. Wagner, Towards evaluating the robustness of neural networks, presented at 2017 IEEE Symp. Security and Privacy (SP), San Jose, CA, USA, 2017, pp. 39-57.
DOI
[36]
N. Dowlin, R. Gilad-Bachrach, K. Laine, K. Lauter, M. Naehrig, and J. Wernsing, Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy, presented at Proc. 33rd Int. Conf. Machine Learning, New York, NY, USA, 2016, pp. 201-210.
[37]
S. Lee, H. Kim, J. Park, J. Jang, C. S. Jeong, and S. Yoon, TensorLightning: A traffic-efficient distributed deep learning on commodity spark clusters, IEEE Access, vol. 6, pp. 27 671-27 680, 2018.
DOI Google Scholar
[38]
K. Eykholt, I. Evtimov, E. Fernandes, B. Li, A. Rahmati, C. W. Xiao, A. Prakash, T. Kohno, and D. Song, Robust physical-world attacks on deep learning visual classification, presented at 2018 IEEE/CVF Conf. Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 1625-1634.
DOI
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 06 September 2020
Accepted: 09 October 2020
Published: 09 June 2021
Issue date: December 2021

Copyright

© The author(s) 2021.

Acknowledgements

This work was supported by the National Key Research and Development Program of China (No. 2018YFB0803403); Fundamental Research Funds for the Central Universities (Nos. FRF-AT-19-009Z and FRF-BD-19-012A), and National Social Science Fund of China (No. 18BGJ071).

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return