Journal Home > Volume 29 , Issue 3

In recent years, Deep Learning (DL) technique has been widely used in Internet of Things (IoT) and Industrial Internet of Things (IIoT) for edge computing, and achieved good performances. But more and more studies have shown the vulnerability of neural networks. So, it is important to test the robustness and vulnerability of neural networks. More specifically, inspired by layer-wise relevance propagation and neural network verification, we propose a novel measurement of sensitive neurons and important neurons, and propose a novel neuron coverage criterion for robustness testing. Based on the novel criterion, we design a novel testing sample generation method, named DeepSI, which involves definitions of sensitive neurons and important neurons. Furthermore, we construct sensitive-decision paths of the neural network through selecting sensitive neurons and important neurons. Finally, we verify our idea by setting up several experiments, then results show our proposed method achieves superior performances.


menu
Abstract
Full text
Outline
About this article

DeepSI: A Sensitive-Driven Testing Samples Generation Method of Whitebox CNN Model for Edge Computing

Show Author's information Zhichao Lian1( )Fengjun Tian1
School of Cyber Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China

Abstract

In recent years, Deep Learning (DL) technique has been widely used in Internet of Things (IoT) and Industrial Internet of Things (IIoT) for edge computing, and achieved good performances. But more and more studies have shown the vulnerability of neural networks. So, it is important to test the robustness and vulnerability of neural networks. More specifically, inspired by layer-wise relevance propagation and neural network verification, we propose a novel measurement of sensitive neurons and important neurons, and propose a novel neuron coverage criterion for robustness testing. Based on the novel criterion, we design a novel testing sample generation method, named DeepSI, which involves definitions of sensitive neurons and important neurons. Furthermore, we construct sensitive-decision paths of the neural network through selecting sensitive neurons and important neurons. Finally, we verify our idea by setting up several experiments, then results show our proposed method achieves superior performances.

Keywords: neuron sensitivity, Layer-wise Relevance Propagation (LRP), neural network verification, deep learning testing

References(60)

[1]
X. Chu, Z. Tian, Y. Wang, B. Zhang, H. Ren, X. Wei, H. Xia, and C. Shen, Twins: Revisiting the design of spatial attention in vision transformers, arXiv preprint arXiv: 2104.13840, 2021.
[2]

Q. Hua, L. Chen, P. Li, S. Zhao, and Y. Li, A pixel-channel hybrid attention model for image processing, Tsinghua Science and Technology, vol. 27, no. 5, pp. 804–816, 2022.

[3]

S. Wu, S. Shen, X. Xu, Y. Chen, X. Zhou, D. Liu, X. Xue, and L. Qi, Popularity-aware and diverse web APIs recommendation based on correlation graph, IEEE Trans. Comput. Soc. Syst., vol. 10, no. 2, pp. 771–782, 2023.

[4]

X. Zhou, Y. Li, and W. Liang, CNN-RNN based intelligent recommendation for online medical pre-diagnosis support, IEEE/ACM Trans. Comput. Biol. Bioinform., vol. 18, no. 3, pp. 912–921, 2021.

[5]
Z. Liu, H. Hu, Y. Lin, Z. Yao, Z. Xie, Y. Wei, J. Ning, Y. Cao, Z. Zhang, L. Dong, et al., Swin transformer V2: Scaling up capacity and resolution, in Proc. 2022 IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 11999–12009.
DOI
[6]

J. Sun, X. Jiang, J. Liu, F. Zhang, and C. Li, An anti-recompression video watermarking algorithm in bitstream domain, Tsinghua Science and Technology, vol. 26, no. 2, pp. 154–162, 2020.

[7]

F. Wang, G. Li, Y. Wang, W. Rafique, M. R. Khosravi, G. Liu, Y. Liu, and L. Qi, Privacy-aware traffic flow prediction based on multi-party sensor data with zero trust in smart city, ACM Trans. Internet Technol., vol. 23, no. 3, pp. 1–19, 2023.

[8]

F. Wang, H. Zhu, G. Srivastava, S. Li, M. R. Khosravi, and L. Qi, Robust collaborative filtering recommendation with user-item-trust records, IEEE Trans. Comput. Soc. Syst., vol. 9, no. 4, pp. 986–996, 2022.

[9]

L. C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs, IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 4, pp. 834–848, 2018.

[10]

L. Zhang, K. Zhang, and H. Pan, SUNet++: A deep network with channel attention for small-scale object segmentation on 3D medical images, Tsinghua Science and Technology, vol. 28, no. 4, pp. 628–638, 2023.

[11]

Y. Xu, C. Zhang, G. Wang, Z. Qin, and Q. Zeng, A blockchain-enabled deduplicatable data auditing mechanism for network storage services, IEEE Trans. Emerg. Top. Comput., vol. 9, no. 3, pp. 1421–1432, 2021.

[12]

Y. Xu, J. Ren, Y. Zhang, C. Zhang, B. Shen, and Y. Zhang, Blockchain empowered arbitrable data auditing scheme for network storage as a service, IEEE Trans. Serv. Comput., vol. 13, no. 2, pp. 289–300, 2020.

[13]
J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, BERT: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv: 1810.04805, 2018.
[14]

F. Wang, L. Wang, G. Li, Y. Wang, C. Lv, and L. Qi, Edge-cloud-enabled matrix factorization for diversified APIs recommendation in mashup creation, World Wide Web, vol. 25, no. 5, pp. 1809–1829, 2022.

[15]

L. Kong, L. Wang, W. Gong, C. Yan, Y. Duan, and L. Qi, LSH-aware multitype health data prediction with privacy preservation in edge environment, World Wide Web, vol. 25, no. 5, pp. 1793–1808, 2022.

[16]
Y. Yang, X. Yang, M. Heidari, M. A. Khan, G. Srivastava, M. Khosravi, and L. Qi, ASTREAM: Data-stream-driven scalable anomaly detection with accuracy guarantee in IIoT environment, IEEE Trans. Netw. Sci. Eng. doi: 10.1109/TNSE.2022.3157730.
DOI
[17]

X. Zhou, W. Liang, W. Li, K. Yan, S. Shimizu, and K. I. K. Wang, Hierarchical adversarial attacks against graph-neural-network-based IoT network intrusion detection system, IEEE Internet Things J., vol. 9, no. 12, pp. 9310–9319, 2022.

[18]

L. Qi, Y. Yang, X. Zhou, W. Rafique, and J. Ma, Fast anomaly identification based on multiaspect data streams for intelligent intrusion detection toward secure industry 4.0, IEEE Trans. Ind. Inform., vol. 18, no. 9, pp. 6503–6511, 2022.

[19]
Z. Cai and Z. He, Trading private range counting over big IoT data, in Proc. 2019 IEEE 39th Int. Conf. Distributed Computing Systems (ICDCS), Dallas, TX, USA, 2019, pp. 144–153.
DOI
[20]
L. Kong, G. Li, W. Rafique, S. Shen, Q. He, M. R. Khosravi, R. Wang, and L. Qi, Time-aware missing healthcare data prediction based on ARIMA model, IEEE/ACM Trans. Comput. Biol. Bioinform. doi: 10.1109/TCBB.2022.3205064.
DOI
[21]

L. Nie, Z. Ning, X. Wang, X. Hu, J. Cheng, and Y. Li, Data-driven intrusion detection for intelligent Internet of vehicles: A deep convolutional neural network-based method, IEEE Trans. Netw. Sci. Eng., vol. 7, no. 4, pp. 2219–2230, 2020.

[22]

Y. Xu, Z. Liu, C. Zhang, J. Ren, Y. Zhang, and X. Shen, Blockchain-based trustworthy energy dispatching approach for high renewable energy penetrated power systems, IEEE Internet Things J., vol. 9, no. 12, pp. 10036–10047, 2022.

[23]

C. Zhang, Y. Xu, Y. Hu, J. Wu, J. Ren, and Y. Zhang, A blockchain-based multi-cloud storage data auditing scheme to locate faults, IEEE Trans. Cloud Comput., vol. 10, no. 4, pp. 2252–2263, 2022.

[24]

X. Zhou, X. Yang, J. Ma, and K. I. K. Wang, Energy-efficient smart routing based on link correlation mining for wireless edge computing in IoT, IEEE Internet Things J., vol. 9, no. 16, pp. 14988–14997, 2022.

[25]

X. Zhou, X. Xu, W. Liang, Z. Zeng, and Z. Yan, Deep-learning-enhanced multitarget detection for end–edge–cloud surveillance in smart IoT, IEEE Internet Things J., vol. 8, no. 16, pp. 12588–12596, 2021.

[26]
K. Pei, Y. Cao, J. Yang, and S. Jana, DeepXplore: Automated whitebox testing of deep learning systems, in Proc. 26th Symp. on Operating Systems Principles, Shanghai, China, 2017, pp. 1–18.
DOI
[27]
L. Ma, F. Juefei-Xu, F. Zhang, J. Sun, M. Xue, B. Li, C. Chen, T. Su, L. Li, Y. Liu, et al., DeepGauge: Multi-granularity testing criteria for deep learning systems, in Proc. 2018 33rd IEEE/ACM Int. Conf. Automated Software Engineering (ASE), Montpellier, France, 2018, pp. 120–131.
DOI
[28]

X. Zhou, W. Liang, K. I. K. Wang, and L. T. Yang, Deep correlation mining based on hierarchical hybrid networks for heterogeneous big data recommendations, IEEE Trans. Comput. Soc. Syst., vol. 8, no. 1, pp. 171–178, 2021.

[29]

L. Qi, W. Lin, X. Zhang, W. Dou, X. Xu, and J. Chen, A correlation graph based approach for personalized and compatible web APIs recommendation in mobile APP development, IEEE Trans. Knowl. Data Eng., vol. 35, no. 6, pp. 5444–5457, 2023.

[30]

S. Bach, A. Binder, G. Montavon, F. Klauschen, K. R. Müller, and W. Samek, On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation, PLoS One, vol. 10, no. 7, p. e0130140, 2015.

[31]
H. Zhang, H. Chen, C. Xiao, S. Gowal, R. Stanforth, B. Li, D. Boning, and C. J. Hsieh, Towards stable and efficient training of verifiably robust neural networks, arXiv preprint arXiv: 1906.06316, 2019.
[32]
R. S. Pressman, Software Engineering : A Practitioner’s Approach. London, UK: Palgrave Macmillan, 2005.
[33]

N. B. Ruparelia, Software development lifecycle models, SIGSOFT Softw. Eng. Notes, vol. 35, no. 3, pp. 8–13, 2010.

[34]
Y. Tian, K. Pei, S. Jana, and B. Ray, DeepTest: Automated testing of deep-neural-network-driven autonomous cars, in Proc. 2018 IEEE/ACM 40th Int. Conf. Software Engineering (ICSE), Gothenburg, Sweden, 2018, pp. 303–314.
DOI
[35]
A. Odena and I. Goodfellow, TensorFuzz: Debugging neural networks with coverage-guided fuzzing, arXiv preprint arXiv: 1807.10875, 2018.
[36]
L. Ma, F. Juefei-Xu, M. Xue, B. Li, L. Li, Y. Liu, and J. Zhao, DeepCT: Tomographic combinatorial testing for deep learning systems, in Proc. 2019 IEEE 26th Int. Conf. Software Analysis, Evolution and Reengineering (SANER), Hangzhou, China, 2019, pp. 614–618.
DOI
[37]
Y. Tian, Z. Zhong, V. Ordonez, and B. Ray, Testing deep neural network based image classifiers, arXiv preprint arXiv: 1905.07831, 2019.
[38]
D. Wang, Z. Wang, C. Fang, Y. Chen, and Z. Chen, DeepPath: Path-driven testing criteria for deep neural networks, in Proc. 2019 IEEE Int. Conf. Artificial Intelligence Testing (AITest), Newark, CA, USA, 2019, pp. 119–120.
DOI
[39]
Z. Zhou, W. Dou, J. Liu, C. Zhang, J. Wei, and D. Ye, DeepCon: Contribution coverage testing for deep learning systems, in Proc. 2021 IEEE Int. Conf. Software Analysis, Evolution and Reengineering (SANER), Honolulu, HI, USA, 2021, pp. 189–200.
DOI
[40]
X. Xie, L. Ma, F. Juefei-Xu, M. Xue, H. Chen, Y. Liu, J. Zhao, B. Li, J. Yin, and S. See, DeepHunter: A coverage-guided fuzz testing framework for deep neural networks, in Proc. 28th ACM SIGSOFT Int. Symp. on Software Testing and Analysis, Beijing, China, 2019, pp. 146–157.
DOI
[41]
G. Katz, C. Barrett, D. Dill, K. Julian, and M. Kochenderfer, Reluplex: An efficient SMT solver for verifying deep neural networks, arXiv preprint arXiv: 1702.01135, 2017.
DOI
[42]
X. Huang, M. Kwiatkowska, S. Wang, and M. Wu, Safety verification of deep neural networks, in Computer Aided Verification, R. Majumdar and V. Kunčak, eds. Cham, Switzerland: Springer, 2017, pp. 3–29.
DOI
[43]
R. Ehlers, Formal verification of piece-wise linear feed-forward neural networks, in Automated Technology for Verification and Analysis, D. D’Souza and K. N. Kumar, eds. Cham, Switzerland: Springer, 2017, pp. 269–286.
DOI
[44]
G. Singh, T. Gehr, M. Mirman, M. Püschel, and M. Vechev, Fast and effective robustness certification, in Proc. 32nd Int. Conf. Neural Information Processing Systems, Montréal, Canada, 2018, pp. 10825–10836.
[45]
G. Singh, T. Gehr, M. Püschel, and M. Vechev, Boosting robustness certification of neural networks, https://openreview.net/forum?id=HJgeEh09KQ, 2018.
[46]
R. Bunel, I. Turkaslan, P. H. S. Torr, P. Kohli, and M. P. Kumar, A unified view of piecewise linear neural network verification, in Proc. 32nd Int. Conf. Neural Information Processing Systems, Montréal, Canada, 2018, pp. 4795–4804.
[47]
C. Müller, F. Serre, G. Singh, M. Püschel, and M. Vechev, Scaling polyhedral neural network verification on GPUs, arXiv preprint arXiv: 2007.10868, 2020.
[48]

G. Singh, T. Gehr, M. Püschel, and M. Vechev, An abstract domain for certifying neural networks, Proc. ACM Program. Lang., vol. 3, no. POPL, p. 41, 2019.

[49]
H. Zhang, T. W. Weng, P. Y. Chen, C. J. Hsieh, and L. Daniel, Efficient neural network robustness certification with general activation functions, in Proc. 32nd Int. Conf. Neural Information Processing Systems, Montréal, Canada, 2018, pp. 4944–4953.
[50]

X. Xie, T. Li, J. Wang, L. Ma, Q. Guo, F. Juefei-Xu, and Y. Liu, NPC: Neuron path coverage via characterizing decision logic of deep neural networks, ACM Trans. Softw. Eng. Methodol., vol. 31, no. 3, p. 47, 2022.

[51]
J. Gu, Y. Yang, and V. Tresp, Understanding individual decisions of CNNs via contrastive backpropagation, in Computer Vision – ACCV 2018, C. V. Jawahar, H. Li, G. Mori, and K. Schindler, eds. Cham, Switzerland: Springer, 2018, pp. 119–134.
DOI
[52]
B. K. Iwana, R. Kuroki, and S. Uchida, Explaining convolutional neural networks using softmax gradient layer-wise relevance propagation, in Proc. 2019 IEEE/CVF Int. Conf. Computer Vision Workshop (ICCVW), Seoul, Republic of Korea, 2019, pp. 4176–4185.
DOI
[53]
K. Xu, H. Zhang, S. Wang, Y. Wang, S. Jana, X. Lin, and C. J. Hsieh, Fast and complete: Enabling complete neural network verification with rapid and massively parallel incomplete verifiers, arXiv preprint arXiv: 2011.13824, 2020.
[54]
S. Wang, H. Zhang, K. Xu, X. Lin, S. Jana, C. J. Hsieh, and J. Z. Kolter, Beta-CROWN: Efficient bound propagation with per-neuron split constraints for complete and incomplete neural network robustness verification, arXiv preprint arXiv: 2103.06624, 2021.
[55]
S. Gowal, K. Dvijotham, R. Stanforth, R. Bunel, C. Qin, J. Uesato, R. Arandjelovic, T. Mann, and P. Kohli, On the effectiveness of interval bound propagation for training verifiably robust models, arXiv preprint arXiv: 1810.12715, 2018.
[56]
K. He, X. Zhang, S. Ren, and J. Sun, Deep residual learning for image recognition, in Proc. 2016 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 770–778.
DOI
[57]
G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, Densely connected convolutional networks, in Proc. 2017 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 2261–2269.
DOI
[58]
Y. LeCun and C. Cortes, The MNIST database of handwritten digits, http://yann.lecun.com/exdb/mnist/, 1998.
[59]
N. Krizhevsky, H. Vinod, C. Geoffrey, M. Papadakis, and A. Ventresque, The CIFAR-10 dataset, http://www.cs.toronto.edu/kriz/cifar.html, 2014.
[60]
J. Ren, M. Li, Z. Liu, and Q. Zhang, Interpreting and disentangling feature components of various complexity from DNNs, arXiv preprint arXiv: 2006.15920, 2020.
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 03 February 2023
Revised: 11 May 2023
Accepted: 02 June 2023
Published: 04 December 2023
Issue date: June 2024

Copyright

© The Author(s) 2024.

Acknowledgements

Acknowledgment

This work was supported by the National Key R&D Program of China (No. 2021YFF0602104-2).

Rights and permissions

The articles published in this open access journal are distributed under the terms of theCreative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return