Journal Home > Volume 2 , Issue 3

Network embedding is a very important task to represent the high-dimensional network in a low-dimensional vector space, which aims to capture and preserve the network structure. Most existing network embedding methods are based on shallow models. However, actual network structures are complicated which means shallow models cannot obtain the high-dimensional nonlinear features of the network well. The recently proposed unsupervised deep learning models ignore the labels information. To address these challenges, in this paper, we propose an effective network embedding method of Structural Labeled Locally Deep Nonlinear Embedding (SLLDNE). SLLDNE is designed to obtain highly nonlinear features through utilizing deep neural network while preserving the label information of the nodes by using a semi-supervised classifier component to improve the ability of discriminations. Moreover, we exploit linear reconstruction of neighborhood nodes to enable the model to get more structural information. The experimental results of vertex classification on two real-world network datasets demonstrate that SLLDNE outperforms the other state-of-the-art methods.


menu
Abstract
Full text
Outline
About this article

A Semi-Supervised Deep Network Embedding Approach Based on the Neighborhood Structure

Show Author's information Wenmao WuZhizhou YuJieyue He( )
School of Computer Science and Engineering, and also with MOE Key Laboratory of Computer Network and Information Integration, Southeast University, Nanjing 211100, China.

Abstract

Network embedding is a very important task to represent the high-dimensional network in a low-dimensional vector space, which aims to capture and preserve the network structure. Most existing network embedding methods are based on shallow models. However, actual network structures are complicated which means shallow models cannot obtain the high-dimensional nonlinear features of the network well. The recently proposed unsupervised deep learning models ignore the labels information. To address these challenges, in this paper, we propose an effective network embedding method of Structural Labeled Locally Deep Nonlinear Embedding (SLLDNE). SLLDNE is designed to obtain highly nonlinear features through utilizing deep neural network while preserving the label information of the nodes by using a semi-supervised classifier component to improve the ability of discriminations. Moreover, we exploit linear reconstruction of neighborhood nodes to enable the model to get more structural information. The experimental results of vertex classification on two real-world network datasets demonstrate that SLLDNE outperforms the other state-of-the-art methods.

Keywords: deep learning, network embedding, network analysis

References(29)

[1]
P. Sen, G. Namata, M. Bilgic, L. Getoor, B. Galligher, and T. Eliassi-Rad, Collective classification in network data, AI Mag., vol. 29, no. 3, pp. 93-106, 2008.
[2]
I. Herman, G. Melancon, and M. S. Marshall, Graph visualization and navigation in information visualization: A survey, IEEE Trans. Vis. Comput. Graph., vol. 6, no. 1, pp. 24-43, 2000.
[3]
D. Liben-Nowell and J. Kleinberg, The link-prediction problem for social networks, J. Amer. Soc. Inf. Sci. Technol., vol. 58, no. 7, pp. 1019-1031, 2007.
[4]
J. B. Tenenbaum, V. De Silva, and J. C. Langford, A global geometric framework for nonlinear dimensionality reduction, Science, vol. 290, no. 5500, pp. 2319-2323, 2000.
[5]
S. T. Roweis and K. S. Lawrence, Nonlinear dimensionality reduction by locally linear embedding, Science, vol. 290, no. 5500, pp. 2323-2326, 2000.
[6]
M. Belkin and P. Niyogi, Laplacian eigenmaps and spectral techniques for embedding and clustering, in Proc. 14th Int. Conf. Neural Information Processing Systems: Natural and Synthetic, Vancouver, Canada, 2001, pp. 585-591.
[7]
M. Chen, Q. Yang, and X. O. Tang, Directed graph embedding, in Proc. 20th Int. Joint Conf. Artificial Intelligence, Hyderabad, India, 2007, pp. 2707-2712.
[8]
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, Distributed representations of words and phrases and their compositionality, in Advances in Neural Information Processing Systems 26, Lake Tahoe, NV, USA, 2013, pp. 3111-3119.
[9]
T. Mikolov, K. Chen, G. Corrado, and J. Dean, Efficient estimation of word representations in vector space, arXiv preprint arXiv: 1301.3781, 2013.
[10]
T. Mikolov, W. T. Yih, and G. Zweig, Linguistic regularities in continuous space word representations, in Proc. 2013 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Atlanta, GA, USA, 2013, pp. 746-751.
[11]
B. Perozzi, R. Al-Rfou, and S. Skiena, DeepWalk: Online learning of social representations, in Proc. 20th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, New York, NY, USA, 2014, pp. 701-710.
DOI
[12]
J. Tang, M. Qu, M. Z. Wang, M. Zhang, J. Yan, and Q. Z. Mei, LINE: Large-scale information network embedding, arXiv preprint arXiv: 1503.03578, 2015.
[13]
A. Grover and J. Leskovec, node2vec: Scalable feature learning for networks, in Proc. 22nd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, San Francisco, CA, USA, 2016, pp. 855-864.
DOI
[14]
T. Tang, M. Qu, and Q. Z. Mei, PTE: Predictive text embedding through large-scale heterogeneous text networks, in Proc. 21st ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Sydney, Australia, 2015, pp. 1165-1174.
DOI
[15]
D. X. Wang, P. Cui, and W. W. Zhu, Structural deep network embedding, in Proc. 22nd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, San Francisco, CA, USA, 2016, pp. 1225-1234.
DOI
[16]
S. S. Cao, L. Wei, and Q. K. Xu, Deep neural networks for learning graph representations, in Proc. 13th AAAI Conf. Artificial Intelligence, Phoenix, AZ, USA, 2016, pp. 1145-1152.
[17]
C. Cortes and V. Vapnik, Support-vector networks, Mach. Learn., vol. 20, no. 3, pp. 273-297, 1995.
[18]
F. Tian, B. Gao, Q. Cui, E. H. Chen, and T. Y. Liu, Learning deep representations for graph clustering, in Proc. 28th AAAI Conf. Artificial Intelligence, Québec City, Canada, 2014, pp. 1293-1299.
[19]
P. Vincent, H. Larochelle, Y. Bengio, and P. A. Manzagol, Extracting and composing robust features with denoising autoencoders, in Proc. 25th Int. Conf. Machine Learning, Helsinki, Finland, 2008, pp. 1096-1103.
DOI
[20]
S. Y. Chang, W. Han, J. L. Tang, G. J. Qi, C. C. Aggarwal, and T. S. Huang, Heterogeneous network embedding via deep architectures, in Proc. 21th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Sydney, Australia, 2015, pp. 119-128.
DOI
[21]
L. Page, S. Brin, R. Motwani, and T. Winograd, The PageRank citation ranking: Bringing order to the web. Stanford InfoLab, Stanford University, Stanford, CA, USA, 1999.
[22]
M. E. J. Newman, Finding community structure in networks using the eigenvectors of matrices, Phys. Rev. E, vol. 74, p. 036104, 2006.
[23]
X. Wang, P. Cui, J. Wang, J. Pei, W. W. Zhu, and S. Q. Yang, Community preserving network embedding, in Proc. 31st AAAI Conf. Artificial Intelligence, San Francisco, CA, USA, 2017, pp. 203-209.
[24]
D. P. Kingma and B. Jimmy, Adam: A method for stochastic optimization, arXiv preprint arXiv: 1412.6980, 2014.
[25]
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, Gradient-based learning applied to document recognition, Proc. IEEE, vol. 86, vol. 11, pp. 2278-2324, 1998.
[26]
A. K. McCallum, K. Nigam, J. Rennie, and K. Seymore, Automating the construction of internet portals with machine learning, Information Retrieval, vol. 3, no. 2, pp. 127-163, 2000.
[27]
X. Zhang, W. Z. Chen, Z. M. Xie, and H. F. Yan, Learning transductive network embedding, J. Front. Comput. Sci. Technol., vol. 11, no. 4, pp. 520-527, 2017.
[28]
C. C. Tu, W. C. Zhang, Z. Y. Liu, and M. S. Sun, Max-Margin DeepWalk: Discriminative learning of network representation, in Proc. 25th Int. Joint Conf. Artificial Intelligence, New York, NY, USA, 2016, pp. 3889-3895.
[29]
R. E. Fan, K. W. Chang, C. J. Hsieh, X. R. Wang, and C. J. Lin, LIBLINEAR: A library for large linear classification, J. Mach. Learn. Res., vol. 9, pp. 1871-1874, 2008.
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 22 November 2018
Accepted: 13 February 2019
Published: 04 April 2019
Issue date: September 2019

Copyright

© The author(s) 2019

Acknowledgements

This work was supported by the National Natural Science Foundation of China (No. 61370207) and Collaborative Innovation Center of Novel Software Technology and Industrialization.

Rights and permissions

Return