AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Journal of Computer Science and Technology Article
Article Link
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Regular Paper

Identity-Preserving Adversarial Training for Robust Network Embedding

Data Intelligence System Research Center, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China
University of Chinese Academy of Sciences, Beijing 101480, China
Beijing Academy of Artificial Intelligence, Beijing 100000, China
Chinese Academy of Sciences Key Laboratory of Network Data Science and Technology, Institute of Computing Technology Chinese Academy of Sciences, Beijing 100190, China
Show Author Information

Abstract

Network embedding, as an approach to learning low-dimensional representations of nodes, has been proved extremely useful in many applications, e.g., node classification and link prediction. Unfortunately, existing network embedding models are vulnerable to random or adversarial perturbations, which may degrade the performance of network embedding when being applied to downstream tasks. To achieve robust network embedding, researchers introduce adversarial training to regularize the embedding learning process by training on a mixture of adversarial examples and original examples. However, existing methods generate adversarial examples heuristically, failing to guarantee the imperceptibility of generated adversarial examples, and thus limit the power of adversarial training. In this paper, we propose a novel method Identity-Preserving Adversarial Training (IPAT) for network embedding, which generates imperceptible adversarial examples with explicit identity-preserving regularization. We formalize such identity-preserving regularization as a multi-class classification problem where each node represents a class, and we encourage each adversarial example to be discriminated as the class of its original node. Extensive experimental results on real-world datasets demonstrate that our proposed IPAT method significantly improves the robustness of network embedding models and the generalization of the learned node representations on various downstream tasks.

Keywords

network embeddingidentity-preservingadversarial trainingadversarial the example

Electronic Supplementary Material

Download File(s)
JCST-2202-12256-Highlights.pdf (300.3 KB)

References

[1]

Cui P, Wang X, Pei J, Zhu W W. A survey on network embedding. IEEE Trans. Knowledge and Data Engineering , 2019, 31(5): 833–852. DOI: 10.1109/TKDE.2018.2849 727.
Crossref Google Scholar
[2]
Qu L, Zhu H S, Zheng R Q, Shi Y H, Yin H Z. ImGAGN: Imbalanced network embedding via generative adversarial graph networks. In Proc. the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, Aug. 2021, pp.1390–1398. DOI: 10.1145/3447548.3467334.
Crossref
[3]

Ruan C Y, Wang Y, Ma J G, Zhang Y C, Chen X T. Adversarial heterogeneous network embedding with metapath attention mechanism. Journal of Computer Science and Technology , 2019, 34(6): 1217–1229. DOI: 10.1007/s11390-019-1971-3.
Crossref Google Scholar
[4]
Hu Z N, Dong Y X, Wang K S, Chang K W, Sun Y Z. GPT-GNN: Generative pre-training of graph neural networks. In Proc. the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Aug. 2020, pp.1857–1867. DOI: 10.1145/3394486.3403237.
Crossref
[5]
Perozzi B, Al-Rfou R, Skiena S. DeepWalk: Online learning of social representations. In Proc. the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Aug. 2014, pp.701–710. DOI: 10.1145/2623330.2623732.
Crossref
[6]
Dai Q Y, Shen X, Zhang L, Li Q, Wang D. Adversarial training methods for network embedding. In Proc. the World Wide Web Conference, May 2019, pp.329–339. DOI: 10.1145/3308558.3313445.
Crossref
[7]
Dai Q Y, Li Q, Tang J, Wang D. Adversarial network embedding. In Proc. the 32nd AAAI Conference on Artificial Intelligence, Feb. 2018, pp.2167–2174. DOI: 10.1609/aaai.v32i1.11865.
Crossref
[8]
Qin Y, Carlini N, Cottrell G W, Goodfellow I J, Raffel C. Imperceptible, robust, and targeted adversarial examples for automatic speech recognition. In Proc. the 36th International Conference on Machine Learning, Jun. 2019, pp.5231–5240.
[9]
Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I J, Fergus R. Intriguing properties of neural networks. In Proc. the 2nd International Conference on Learning Representations, Apr. 2014.
[10]
Kipf T N, Welling M. Variational graph autoencoders. In Proc. the NIPS Workshop on Bayesian Deep Learning, Dec. 2016.
[11]
Veličković P, Fedus W, Hamilton W L, Liò P, Bengio Y, Hjelm R D. Deep graph infomax. In Proc. the 7th International Conference on Learning Representations, May 2019.
[12]
Grover A, Leskovec J. node2vec: Scalable feature learning for networks. In Proc. the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Aug. 2016, pp.855–864. DOI: 10.1145/2939672.2939754.
Crossref
[13]
Wang D X, Cui P, Zhu W W. Structural deep network embedding. In Proc. the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Aug. 2016, pp.1225–1234. DOI: 10.1145/2939672.2939753.
Crossref
[14]
Liu J, He Z C, Wei L, Huang Y L. Content to node: Self-translation network embedding. In Proc. the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Aug. 2018, pp.1794–1802. DOI: 10.1145/3219819.3219988.
Crossref
[15]
Gao H C, Huang H. Deep attributed network embedding. In Proc. the 27th International Joint Conference on Artificial Intelligence, Jul. 2018, pp.3364–3370. DOI: 10.24963/ijcai.2018/467.
Crossref
[16]
Goodfellow I J, Shlens J, Szegedy C. Explaining and harnessing adversarial examples. In Proc. the 3rd International Conference on Learning Representations, May 2015.
[17]
Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks. In Proc. the 5th International Conference on Learning Representations, Apr. 2017.
[18]
Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y. Graph attention networks. In Proc. the 6th International Conference on Learning Representations, Apr. 2018.
[19]

Sen P, Namata G, Bilgic M, Getoor L, Gallagher B, Eliassi-Rad T. Collective classification in network data. AI Magazine , 2008, 29(3): 93–106. DOI: 10.1609/aimag.v29i3.2157.
Crossref Google Scholar
[20]

Golub G H, Reinsch C. Singular value decomposition and least squares solutions. Numerische Mathematik , 1970, 14(5): 403–420. DOI: 10.1007/BF02163027.
Crossref Google Scholar
[21]
You Y N, Chen T L, Sui Y D, Chen T, Wang Z Y, Shen Y. Graph contrastive learning with augmentations. In Proc. the 34th International Conference on Neural Information Processing Systems, Dec. 2020.
[22]
Pan S R, Hu R Q, Long G D, Jiang J, Yao L N, Zhang C Q. Adversarially regularized graph autoencoder for graph embedding. In Proc. the 27th International Joint Conference on Artificial Intelligence, Jul. 2018, pp.2609–2615. DOI: 10.24963/ijcai.2018/362.
Crossref
[23]
Yang C, Liu Z Y, Zhao D L, Sun M S, Chang E Y. Network representation learning with rich text information. In Proc. the 24th International Conference on Artificial Intelligence, Jun. 2015, pp.2111–2117. DOI: 10.5555/2832415.2832542.
[24]
Abadi M, Barham P, Chen J M, Chen Z F, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M, Kudlur M, Levenberg J, Monga R, Moore S, Murray D G, Steiner B, Tucker P, Vasudevan V, Warden P, Wicke M, Yu Y, Zheng X Q. Tensorflow: A system for large-scale machine learning. In Proc. the 12th USENIX Conference on Operating Systems Design and Implementation, Jun. 2016, pp.265–283. DOI: 10.5555/3026877.3026899.
[25]

Noble W S. What is a support vector machine? Nature Biotechnology , 2006, 24(12): 1565–1567. DOI: 10.1038/nbt 1206-1565.
Crossref Google Scholar
[26]

Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, VanderPlas J T, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay É. Scikit-learn: Machine learning in Python. The Journal of Machine Learning Research , 2011, 12: 2825–2830. DOI: 10.5555/1953048.2078195.
Crossref Google Scholar
[27]
Yang Z L, Cohen W W, Salakhutdinov R. Revisiting semi-supervised learning with graph embeddings. In Proc. the 33rd International Conference on Machine Learning, Jun. 2016, pp.40–48.
[28]
Hu W H, Fey M, Zitnik M, Dong Y X, Ren H Y, Liu B W, Catasta M, Leskovec J. Open graph benchmark: Datasets for machine learning on graphs. In Proc. the 34th International Conference on Neural Information Processing Systems, Dec. 2020. DOI: 10.5555/2999792.2999959.
[29]

Van Der Maaten L, Hinton G. Visualizing data using t-SNE. Journal of Machine Learning Research , 2008, 9(86): 2579–2605.
Google Scholar
[30]
Xu B B, Shen H W, Cao Q, Qiu Y Q, Cheng X Q. Graph wavelet neural network. In Proc. the 7th International Conference on Learning Representations, Apr. 2019.
[31]
Chen T, Kornblith S, Norouzi M, Hinton G E. A simple framework for contrastive learning of visual representations. In Proc. the 37th International Conference on Machine Learning, Jul. 2020, Article No. 149.
[32]
Qiu J Z, Chen Q B, Dong Y X, Zhang J, Yang H X, Ding M, Wang K S, Tang J. GCC: Graph contrastive coding for graph neural network pre-training. In Proc. the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Aug. 2020, pp.1150–1160. DOI: 10.1145/3394486.3403168.
Crossref
[33]
Hassani K, Khasahmadi A H. Contrastive multi-view representation learning on graphs. In Proc. the 37th International Conference on Machine Learning, Jul. 2020, pp.4116–4126. DOI: 10.5555/3524938.3525323.
[34]
Goodfellow I J, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial nets. In Proc. the 27th International Conference on Neural Information Processing Systems, Dec. 2014, pp.2672–2680.
[35]
Wang H W, Wang J, Wang J L, Zhao M, Zhang W N, Zhang F Z, Xie X, Guo M Y. GraphGAN: Graph representation learning with generative adversarial nets. In Proc. the 32nd AAAI Conference on Artificial Intelligence, Feb. 2018, pp.2508–2515. DOI: 10.1609/aaai.v32i1.11872.
Crossref
[36]
Yu W C, Zheng C, Cheng W, Aggarwal C C, Song D L, Zong B, Chen H F, Wang W. Learning deep network representations with adversarially regularized autoencoders. In Proc. the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Aug. 2018, pp.2663–2671. DOI: 10.1145/3219819.3220000.
Crossref
[37]
Qin C L, Martens J, Gowal S, Krishnan D, Dvijotham K, Fawzi A, De S, Stanforth R, Kohli P. Adversarial robustness through local linearization. In Proc. the 33rd International Conference on Neural Information Processing Systems, Dec. 2019, pp.13824–13833.
[38]
Miyato T, Dai A M, Goodfellow I. Adversarial training methods for semi-supervised text classification. In Proc. the 5th International Conference on Learning Representations, Apr. 2017.
[39]
Zhu D Y, Zhang Z W, Cui P, Zhu W W. Robust graph convolutional networks against adversarial attacks. In Proc. the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Aug. 2019, pp.1399–1407. DOI: 10.1145/3292500.3330851.
Crossref
[40]

Hu W B, Chen C, Chang Y M, Zheng Z B, Du Y F. Robust graph convolutional networks with directional graph adversarial training. Applied Intelligence , 2021, 51(11): 7812–7826. DOI: 10.1007/s10489-021-02272-y.
Crossref Google Scholar
Journal of Computer Science and Technology
Volume 39 Issue 1,
January 2024
Pages 177-191
DOI: 10.1007/s11390-023-2256-4
Cite this article:
Cen K-T, Shen H-W, Cao Q, et al. Identity-Preserving Adversarial Training for Robust Network Embedding. Journal of Computer Science and Technology, 2024, 39(1): 177-191. https://doi.org/10.1007/s11390-023-2256-4

267

Views

0

Crossref

0

Web of Science

0

Scopus

0

CSCD

Altmetrics
Received: 21 February 2022
Accepted: 17 April 2023
Published: 25 January 2024
© Institute of Computing Technology, Chinese Academy of Sciences 2024
Return