Journal Home > Volume 7 , Issue 1
Big Data Mining and Analytics 2024, 7(1): 55-74 https://doi.org/10.26599/BDMA.2023.9020004
Open Access | Issue | Published: 25 December 2023

Discriminatively Constrained Semi-Supervised Multi-View Nonnegative Matrix Factorization with Graph Regularization

Show Author's Information Guosheng Cui1,  Ye Li1( ),  Jianzhong Li3,  Jianping Fan2
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China, and also with Joint Engineering Research Center for Health Big Data Intelligent Analysis Technology, Shenzhen 518055, China
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China, and also with University of Chinese Academy of Sciences, Beijing 100049, China
School of Computer Science and Control Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
Keywords:
multi-view, semi-supervised clustering, discriminative information, geometric information, feature normalizing strategy
Cite this article:
Cui G, Li Y, Li J, et al. Discriminatively Constrained Semi-Supervised Multi-View Nonnegative Matrix Factorization with Graph Regularization. Big Data Mining and Analytics, 2024, 7(1): 55-74. https://doi.org/10.26599/BDMA.2023.9020004
Download citation
EndNote(RIS)
BibTeX

287

Views

51

Downloads

Citations

0

Crossref

0

WoS

0

Scopus

0

CSCD

Abstract Full text About this article

Nonnegative Matrix Factorization (NMF) is one of the most popular feature learning technologies in the field of machine learning and pattern recognition. It has been widely used and studied in the multi-view clustering tasks because of its effectiveness. This study proposes a general semi-supervised multi-view nonnegative matrix factorization algorithm. This algorithm incorporates discriminative and geometric information on data to learn a better-fused representation, and adopts a feature normalizing strategy to align the different views. Two specific implementations of this algorithm are developed to validate the effectiveness of the proposed framework: Graph regularization based Discriminatively Constrained Multi-View Nonnegative Matrix Factorization (GDCMVNMF) and Extended Multi-View Constrained Nonnegative Matrix Factorization (ExMVCNMF). The intrinsic connection between these two specific implementations is discussed, and the optimization based on multiply update rules is presented. Experiments on six datasets show that the effectiveness of GDCMVNMF and ExMVCNMF outperforms several representative unsupervised and semi-supervised multi-view NMF approaches.


menu
Abstract
Full text
Outline
About this article

Discriminatively Constrained Semi-Supervised Multi-View Nonnegative Matrix Factorization with Graph Regularization

Show Author's information Guosheng Cui1, Ye Li1( ), Jianzhong Li3, Jianping Fan2
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China, and also with Joint Engineering Research Center for Health Big Data Intelligent Analysis Technology, Shenzhen 518055, China
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China, and also with University of Chinese Academy of Sciences, Beijing 100049, China
School of Computer Science and Control Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China

Abstract

Nonnegative Matrix Factorization (NMF) is one of the most popular feature learning technologies in the field of machine learning and pattern recognition. It has been widely used and studied in the multi-view clustering tasks because of its effectiveness. This study proposes a general semi-supervised multi-view nonnegative matrix factorization algorithm. This algorithm incorporates discriminative and geometric information on data to learn a better-fused representation, and adopts a feature normalizing strategy to align the different views. Two specific implementations of this algorithm are developed to validate the effectiveness of the proposed framework: Graph regularization based Discriminatively Constrained Multi-View Nonnegative Matrix Factorization (GDCMVNMF) and Extended Multi-View Constrained Nonnegative Matrix Factorization (ExMVCNMF). The intrinsic connection between these two specific implementations is discussed, and the optimization based on multiply update rules is presented. Experiments on six datasets show that the effectiveness of GDCMVNMF and ExMVCNMF outperforms several representative unsupervised and semi-supervised multi-view NMF approaches.

Keywords: multi-view, semi-supervised clustering, discriminative information, geometric information, feature normalizing strategy

References(46)

[1]
H. Liang, C. Jiang, D. Feng, X. Chen, H. Xu, X. Liang, W. Zhang, Z. Li, and L. Van Gool, Exploring geometry-aware contrast and clustering harmonization for self-supervised 3D object detection, in Proc. IEEE/CVF Int. Conf. Computer Vision, Montreal, Canada, 2021, pp. 3293–3302.
DOI
[2]

J. Yin and S. Sun, Incomplete multi-view clustering with cosine similarity, Patt. Recogn., vol. 123, p. 108371, 2022.
DOI Google Scholar
[3]

D. Wu, X. Dong, F. Nie, R. Wang, and X. Li, An attention-based framework for multi-view clustering on Grassmann manifold, Patt. Recogn., vol. 128, p. 108610, 2022.
DOI Google Scholar
[4]

Y. Chen, X. Xiao, C. Peng, G. Lu, and Y. Zhou, Low-rank tensor graph learning for multi-view subspace clustering, IEEE Trans. Circ. Syst. Video Technol., vol. 32, no. 1, pp. 92–104, 2022.
DOI Google Scholar
[5]
S. Zhou, X. Liu, J. Liu, X. Guo, Y. Zhao, E. Zhu, Y. Zhai, J. Yin, and W. Gao, Multi-view spectral clustering with optimal neighborhood Laplacian matrix, in Proc. AAAI Conf. Artificial Intelligence, Palo Alto, CA, USA, pp. 6965–6972, 2020.
DOI
[6]

Y. Li, M. Yang, and Z. Zhang, A survey of multi-view representation learning, IEEE Trans. Knowl. Data Eng., vol. 31, no. 10, pp. 1863–1883, 2019.
DOI Google Scholar
[7]

X. Yan, Y. Ye, X. Qiu, and H. Yu, Synergetic information bottleneck for joint multi-view and ensemble clustering, Inform. Fusion, vol. 56, pp 15–27, 2020.
DOI Google Scholar
[8]
B. Cui, H. Yu, T. Zhang, and S. Li, Self-weighted multi-view clustering with deep matrix factorization, in Proc. Eleventh Asian Conf. Machine Learning, Nagoya, Japan, 2019, pp. 567–582.
[9]
J. Xu, H. Tang, Y. Ren, L. Peng, X. Zhu, and L. He, Multi-level feature learning for contrastive multi-view clustering, in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, New Orleans, LA, USA, 2022, pp. 16051–16060.
DOI
[10]

J. Han, J. Xu, F. Nie, and X. Li, Multi-view k-means clustering with adaptive sparse memberships and weight allocation, IEEE Trans. Knowl. Data Eng., vol. 34, no. 2, pp. 816–827, 2022.
DOI Google Scholar
[11]

N. Zhao and J. Bu, Robust multi-view subspace clustering based on consensus representation and orthogonal diversity, Neural Netw., vol. 150, pp. 102–111, 2022.
DOI Google Scholar
[12]
S. Wei, J. Wang, G. Yu, C. Domeniconi, and X. Zhang, Multi-view multiple clusterings using deep matrix factorization, in Proc. AAAI Conf. Artificial Intelligence, Palo Alto, CA, USA, vol. 34, pp. 6348–6355, 2020.
DOI
[13]

J. Liu, F. Cao, and J. Liang, Centroids-guided deep multi-view K-means clustering, Inform. Sci., vol. 609, pp. 876–896, 2022.
DOI Google Scholar
[14]

X. Si, Q. Yin, X. Zhao, and L. Yao, Consistent and diverse multi-view subspace clustering with structure constraint, Patt. Recogn., vol. 121, p. 108196, 2022.
DOI Google Scholar
[15]

L. Hu, N. Wu, and X. Li, Feature nonlinear transformation non-negative matrix factorization with Kullback-Leibler divergence, Patt. Recogn., vol. 132, p. 108906, 2022.
DOI Google Scholar
[16]
J. Liu, C. Wang, J. Gao, and J. Han, Multi-view clustering via joint nonnegative matrix factorization, in Proc. 2013 SIAM Int. Conf. Data Mining, Philadelphia, PA, USA, 2013, pp. 252–260.
DOI
[17]

G. Cui, R. Wang, D. Wu, and Y. Li, Incomplete multiview clustering using normalizing alignment strategy with graph regularization, IEEE Trans. Knowl. Data Eng., vol. 35, no. 8, pp. 8126–8142, 2023.
Google Scholar
[18]

G. Chao, S. Sun, and J. Bi, A survey on multiview clustering, IEEE Trans. Artif. Intell., vol. 2, no. 2, pp. 146–168, 2021.
DOI Google Scholar
[19]

G. A. Khan, J. Hu, T. Li, B. Diallo, and H. Wang, Multi-view data clustering via non-negative matrix factorization with manifold regularization, Int. J. Mach. Learn. Cybernet., vol. 13, no. 3, pp. 677–689, 2022.
DOI Google Scholar
[20]

X. Wang, T. Zhang, and X. Gao, Multiview clustering based on non-negative matrix factorization and pairwise measurements, IEEE Trans. Cybern., vol. 49, no. 9, pp. 3333–3346, 2019.
DOI Google Scholar
[21]

Z. Yang, N. Liang, W. Yan, Z. Li, and S. Xie, Uniform distribution non-negative matrix factorization for multiview clustering, IEEE Trans. Cybern., vol. 51, no. 6, pp. 3249–3262, 2021.
DOI Google Scholar
[22]

Z. Yang, Y. Xiang, K. Xie, and Y. Lai, Adaptive method for nonsmooth nonnegative matrix factorization, IEEE Trans. Neural Netw. Learn. Syst., vol. 28, no. 4, pp. 948–960, 2017.
DOI Google Scholar
[23]

J. Wang, F. Tian, H. Yu, C. H. Liu, K. Zhan, and X. Wang, Diverse non-negative matrix factorization for multiview data representation, IEEE Trans. Cybern., vol. 48, no. 9, pp. 2620–2632, 2018.
DOI Google Scholar
[24]

F. Chen, G. Li, S. Wang, and Z. Pan, Multiview clustering via robust neighboring constraint nonnegative matrix factorization, Math. Probl Eng., vol. 2019, p. 6084382, 2019.
DOI Google Scholar
[25]

G. Du, L. Zhou, K. Lü, and H. Ding, Deep multiple non-negative matrix factorization for multi-view clustering, Intell. Data Anal., vol. 25, no. 2, pp. 339–357, 2021.
DOI Google Scholar
[26]

G. Trigeorgis, K. Bousmalis, S. Zafeiriou, and B. W. Schuller, A deep matrix factorization method for learning attribute representations, IEEE Trans. Patt. Anal. Mach. Intell., vol. 39, no. 3, pp. 417–429, 2017.
DOI Google Scholar
[27]
H. Zhao, Z. Ding, and Y. Fu, Multi-view clustering via deep matrix factorization, in Proc. Thirty-First AAAI Conf. Artificial Intelligence, San Francisco, CA, USA, 2017, pp. 2921–2927.
DOI
[28]

S. Huang, Z. Kang, and Z. Xu, Auto-weighted multi-view clustering via deep matrix decomposition, Patt. Recogn., vol. 97, p. 107015, 2020.
DOI Google Scholar
[29]

K. Luong, R. Nayak, T. Balasubramaniam, and M. A. Bashar, Multi-layer manifold learning for deep non-negative matrix factorization-based multi-view clustering, Patt. Recogn., vol. 131, p.108815, 2022.
DOI Google Scholar
[30]

Y. Jiang, J. Liu, Z. Li, and H. Lu, Semi-supervised unified latent factor learning with multi-view data, Mach. Vision Appl., vol. 25, no. 7, pp. 1635–1645, 2014.
DOI Google Scholar
[31]

J. Liu, Y. Jiang, Z. Li, Z. H. Zhou, and H. Lu, Partially shared latent factor learning with multiview data, IEEE Trans. Neural Netw. Learn. Syst., vol. 26, no. 6, pp. 1233–1246, 2015.
DOI Google Scholar
[32]

N. Liang, Z. Yang, Z. Li, S. Xie, and C. Y. Su, Semi-supervised multi-view clustering with graph-regularized partially shared non-negative matrix factorization, Knowl. Based Syst., vol. 190, p. 105185, 2020.
DOI Google Scholar
[33]
J. Wang, X. Wang, F. Tian, C. H. Liu, H. Yu, and Y. Liu, Adaptive multi-view semi-supervised nonnegative matrix factorization, in Proc. 23 rd Int. Conf. Neural Information Processing, Kyoto, Japan, 2016, pp. 435–444.
DOI
[34]

H. Cai, B. Liu, Y. Xiao, and L. Lin, Semi-supervised multi-view clustering based on constrained nonnegative matrix factorization, Knowl. Based Syst., vol. 182, p. 104798, 2019.
DOI Google Scholar
[35]

H. Cai, B. Liu, Y. Xiao, and L. Lin, Semi-supervised multi-view clustering based on orthonormality-constrained nonnegative matrix factorization, Inform. Sci., vol. 536, pp. 171–184, 2020.
DOI Google Scholar
[36]

S. Wang, J. Cao, F. Lei, Q. Dai, S. Liang, and B. W. K. Ling, Semi-supervised multi-view clustering with weighted anchor graph embedding, Comput. Intell. Neurosci., vol. 2021, p. 4296247, 2021.
DOI Google Scholar
[37]

F. Nie, G. Cai, J. Li, and X. Li, Auto-weighted multi-view learning for image clustering and semi-supervised classification, IEEE Trans. Image Process., vol. 27, no. 3, pp. 1501–1511, 2018.
DOI Google Scholar
[38]

N. Liang, Z. Yang, Z. Li, S. Xie, and W. Sun, Semi-supervised multi-view learning by using label propagation based non-negative matrix factorization, Knowl. Based Syst., vol. 228, p. 107244, 2021.
DOI Google Scholar
[39]

W. Zhao, C. Xu, Z. Guan, and Y. Liu, Multiview concept learning via deep matrix factorization, IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 2, pp. 814–825, 2021.
DOI Google Scholar
[40]

R. Chen, Y. Tang, W. Zhang, and W. Feng, Deep multi-view semi-supervised clustering with sample pairwise constraints, Neurocomputing, vol. 500, pp. 832–845, 2022.
DOI Google Scholar
[41]

H. Liu, Z. Wu, X. Li, D. Cai, and T. S. Huang, Constrained nonnegative matrix factorization for image representation, IEEE Trans. Patt. Anal. Mach. Intell., vol. 34, no. 7, pp. 1299–1311, 2012.
DOI Google Scholar
[42]

Z. Li, J. Tang, and X. He, Robust structured nonnegative matrix factorization for image representation, IEEE Trans. Neural Netw. Learn. Syst., vol. 29, no. 5, pp. 1947–1960, 2018.
DOI Google Scholar
[43]
H. Huang, Y. Luo, G. Zhou, and Q. Zhao, Multi-view data representation via deep autoencoder-like nonnegative matrix factorization, in IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP ), Singapore, 2022, pp. 3338–3342.
DOI
[44]
D. Cai, X. He, X. Wu, and J. Han, Non-negative matrix factorization on manifold, in Proc. 8th IEEE Int. Conf. Data Mining, Pisa, Italy, 2008, pp. 63–72.
DOI
[45]

G. Cui and Y. Li, Nonredundancy regularization based nonnegative matrix factorization with manifold learning for multiview data representation, Informa. Fusion, vol. 82, pp. 86–98, 2022.
DOI Google Scholar
[46]

M. You, A. Yuan, M. Zou, D. Jian He, and X. Li, Robust unsupervised feature selection via multi-group adaptive graph representation, IEEE Trans. Knowl. Data Eng., vol. 35, no. 3, pp. 3030–3044, 2023.
DOI Google Scholar
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 24 November 2022
Revised: 06 March 2023
Accepted: 04 April 2023
Published: 25 December 2023
Issue date: March 2024

Copyright

© The author(s) 2023.

Acknowledgements

Acknowledgment

This work was supported by the National Key Research and Development Project of China (No. 2019YFB2102500), the Strategic Priority CAS Project (No. XDB38040200), the National Natural Science Foundation of China (Nos. 62206269 and U1913210), the Guangdong Provincial Science and Technology Projects (Nos. 2022A1515011217 and 2022A1515011557), and the Shenzhen Science and Technology Projects (No. JSGG20211029095546003).

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return