Journal Home > Volume 29 , Issue 2
Tsinghua Science and Technology 2024, 29(2): 529-542 https://doi.org/10.26599/TST.2023.9010043
Open Access | Issue | Published: 22 September 2023

False Negative Sample Detection for Graph Contrastive Learning

Show Author's Information Binbin Zhang1,  Li Wang1( )
College of Data Science, Taiyuan University of Technology, Jinzhong 030600, China
Keywords:
contrastive learning, graph representation learning, false negative sample detection
Cite this article:
Zhang B, Wang L. False Negative Sample Detection for Graph Contrastive Learning. Tsinghua Science and Technology, 2024, 29(2): 529-542. https://doi.org/10.26599/TST.2023.9010043
Download citation
EndNote(RIS)
BibTeX

71

Views

15

Downloads

Citations

0

Crossref

0

WoS

0

Scopus

0

CSCD

Abstract Full text About this article

Recently, self-supervised learning has shown great potential in Graph Neural Networks (GNNs) through contrastive learning, which aims to learn discriminative features for each node without label information. The key to graph contrastive learning is data augmentation. The anchor node regards its augmented samples as positive samples, and the rest of the samples are regarded as negative samples, some of which may be positive samples. We call these mislabeled samples as "false negative" samples, which will seriously affect the final learning effect. Since such semantically similar samples are ubiquitous in the graph, the problem of false negative samples is very significant. To address this issue, the paper proposes a novel model, False negative sample Detection for Graph Contrastive Learning (FD4GCL), which uses attribute and structure-aware to detect false negative samples. Experimental results on seven datasets show that FD4GCL outperforms the state-of-the-art baselines and even exceeds several supervised methods.


menu
Abstract
Full text
Outline
About this article

False Negative Sample Detection for Graph Contrastive Learning

Show Author's information Binbin Zhang1, Li Wang1( )
College of Data Science, Taiyuan University of Technology, Jinzhong 030600, China

Abstract

Recently, self-supervised learning has shown great potential in Graph Neural Networks (GNNs) through contrastive learning, which aims to learn discriminative features for each node without label information. The key to graph contrastive learning is data augmentation. The anchor node regards its augmented samples as positive samples, and the rest of the samples are regarded as negative samples, some of which may be positive samples. We call these mislabeled samples as "false negative" samples, which will seriously affect the final learning effect. Since such semantically similar samples are ubiquitous in the graph, the problem of false negative samples is very significant. To address this issue, the paper proposes a novel model, False negative sample Detection for Graph Contrastive Learning (FD4GCL), which uses attribute and structure-aware to detect false negative samples. Experimental results on seven datasets show that FD4GCL outperforms the state-of-the-art baselines and even exceeds several supervised methods.

Keywords: contrastive learning, graph representation learning, false negative sample detection

References(35)

[1]
J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, Neural message passing for Quantum chemistry, in Proc. 34th Int. Conf. Machine Learning, Sydney, Australia, 2017, pp. 1263–1272.
Google Scholar
[2]
T. N. Kipf and M. Welling, Semi-supervised classification with graph convolutional networks, presented at 5th Int. Conf. Learning Representations, Toulon, France, 2017.
Google Scholar
[3]
Y. Li, D. Tarlow, M. Brockschmidt, and R. S. Zemel, Gated graph sequence neural networks, presented at 4th Int. Conf. Learning Representations, San Juan, Puerto Rico, 2016.
Google Scholar
[4]
P. Veličović, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, Graph attention networks, presented at 8th Int. Conf. Learning Representations, Vancouver, Canada, 2018.
Google Scholar
[5]
K. Xu, W. Hu, J. Leskovec, and S. Jegelka, How powerful are graph neural networks? presented at 7th Int. Conf. Learning Representations, New Orleans, LA, USA, 2019.
Google Scholar
[6]
H. Zeng, H. Zhou, A. Srivastava, R. Kannan, and V. K. Prasanna, GraphSAINT: Graph sampling based inductive learning method, presented at 8th Int. Conf. Learning Representations, Addis Ababa, Ethiopia, 2020.
Google Scholar
[7]
X. Su, S. Xue, F. Liu, J. Wu, J. Yang, C. Zhou, W. Hu, C. Paris, S. Nepal, D. Jin, et al., A comprehensive survey on community detection with deep learning, IEEE Trans. Neural Netw. Learn. Syst., .
DOI Google Scholar
[8]
M. Zhang and Y. Chen, Link prediction based on graph neural networks, in Proc. 32nd Int. Conf. Neural Information Processing Systems, Montréal, Canada, 2018, pp. 5171–5181.
Google Scholar
[9]
F. Errica, M. Podda, D. Bacciu, and A. Micheli, A fair comparison of graph neural networks for graph classification, presented at 8th Int. Conf. Learning Representations, Addis Ababa, Ethiopia, 2020.
Google Scholar
[10]
F. Y. Sun, J. Hoffmann, V. Verma, and J. Tang, InfoGraph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization, presented at 8th Int. Conf. Learning Representations, Addis Ababa, Ethiopia, 2020.
Google Scholar
[11]
P. Bachman, R. D. Hjelm, and W. Buchwalter, Learning representations by maximizing mutual information across views, in Proc. 33rd Int. Conf. Neural Information Processing Systems, Vancouver, Canada, 2019, pp. 15535–15545.
Google Scholar
[12]
K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, Momentum contrast for unsupervised visual representation learning, in Proc. 2020 IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 9726–9735.
DOI Google Scholar
[13]
Y. Tian, D. Krishnan, and P. Isola, Contrastive multiview coding, in Proc. 16th European Conf. Computer Vision – ECCV 2020, Glasgow, UK, 2020, pp. 776–794.
DOI Google Scholar
[14]
R. Collobert and J. Weston, A unified architecture for natural language processing: Deep neural networks with multitask learning, in Proc. 25th Int. Conf. Machine learning, Helsinki, Finland, 2008, pp. 160–167.
DOI Google Scholar
[15]
A. Mnih and K. Kavukcuoglu, Learning word embeddings efficiently with noise contrastive estimation, in Proc. 26th Int. Conf. Neural Information Processing Systems, Lake Tahoe, NV, USA, 2013, pp. 2265–2273.
Google Scholar
[16]
P. Veličkovic̀, W. Fedus, W. L. Hamilton, P. Liò, Y. Bengio, and R. D. Hjelm, Deep graph infomax, presented at 7th Int. Conf. Learning Representations, New Orleans, LA, USA, 2019.
Google Scholar
[17]
K. Hassani and A. H. Khasahmadi, Contrastive multi-view representation learning on graphs, in Proc. 37th Int. Conf. Machine Learning, Virtual Event, 2020, pp. 4116–4126.
Google Scholar
[18]
Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, and L. Wang, Deep graph contrastive representation learning, arXiv preprint arXiv: 2006.04131, 2020.
Google Scholar
[19]
Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, and L. Wang, Graph contrastive learning with adaptive augmentation, in Proc. Web Conference 2021, Ljubljana, Slovenia, 2021, pp. 2069–2080.
DOI Google Scholar
[20]
T. S. Chen, W. C. Hung, H. Y. Tseng, S. Y. Chien, and M. H. Yang, Incremental false negative detection for contrastive learning, presented at 10th Int. Conf. Learning Representations, Virtual Event, 2022.
Google Scholar
[21]
Z. Hou, X. Liu, Y. Cen, Y. Dong, H. Yang, C. Wang, and J. Tang, GraphMAE: Self-supervised masked graph autoencoders, in Proc. 28th ACM SIGKDD Conf. Knowledge Discovery and Data Mining, Washington, DC, USA, 2022, pp. 594–604.
DOI Google Scholar
[22]
N. Lee, J. Lee, and C. Park, Augmentation-free self-supervised learning on graphs, Proc. AAAI Conf. Artif. Intell., vol. 36, no. 7, pp. 7372–7380, 2022.
DOI Google Scholar
[23]
X. Gong, C. Yang, and C. Shi, MA-GCL: Model augmentation tricks for graph contrastive learning, arXiv preprint arXiv: 2212.07035, 2022.
Google Scholar
[24]
J. Xia, L. Wu, G. Wang, J. Chen, and S. Z. Li, ProGCL: Rethinking hard negative mining in graph contrastive learning, presented at 39th Int. Conf. Machine Learning, Baltimore, MD, USA, 2022.
Google Scholar
[25]
C. Y. Chuang, J. Robinson, Y. C. Lin, A. Torralba, and S. Jegelka, Debiased contrastive learning, presented at 34th Conf. Neural Information Processing Systems, Vancouver, Canada, 2020.
Google Scholar
[26]
T. Huynh, S. Kornblith, M. R. Walter, M. Maire, and M. Khademi, Boosting contrastive self-supervised learning with false negative cancellation, arXiv preprint arXiv: 2011.11765, 2020.
Google Scholar
[27]
P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan, Supervised contrastive learning, presented at 34th Conf. Neural Information Processing Systems, Vancouver, Canada, 2020.
Google Scholar
[28]
X. Qin, N. Sheikh, B. Reinwald, and L. Wu, Relation-aware graph attention model with adaptive self-adversarial training, Proc. AAAI Conf. Artif. Intell., vol. 35, no. 11, pp. 9368–9376, 2021.
DOI Google Scholar
[29]
M. Caron, P. Bojanowski, A. Joulin, and M. Douze, Deep clustering for unsupervised learning of visual features, in Proc. 15th European Conf. Computer Vision, Munich, Germany, 2018, pp. 139–156.
DOI Google Scholar
[30]
J. Yang, D. Parikh, and D. Batra, Joint unsupervised learning of deep representations and image clusters, in Proc. 2016 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 5147–5156.
DOI Google Scholar
[31]
J. Xie, R. Girshick, and A. Farhadi, Unsupervised deep embedding for clustering analysis, in Proc. 33rd Int. Conf.Machine Learning , New York, NY, USA, 2016, pp. 478–487.
Google Scholar
[32]
M. Caron, P. Bojanowski, J. Mairal, and A. Joulin, Unsupervised pre-training of image features on non-curated data, in Proc. 2019 IEEE/CVF Int. Conf. Computer Vision (ICCV), Seoul, Republic of Korea, 2019, pp. 2959–2968.
DOI Google Scholar
[33]
H. Zhao, X. Yang, Z. Wang, E. Yang, and C. Deng, Graph debiased contrastive learning with joint representation clustering, in Proc. Thirtieth Int. Joint Conf. Artificial Intelligence, Montreal, Canada, 2021, pp. 3434–3440.
DOI Google Scholar
[34]
D. Xu, W. Cheng, D. Luo, H. Chen, and X. Zhang, InfoGCL: Information-aware graph contrastive learning, presented at 35th Conf. Neural Information Processing Systems, Virtual Event, 2021.
Google Scholar
[35]
D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, presented at 3rd Int. Conf. Learning Representations, San Diego, CA, USA, 2015.
Google Scholar
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 02 February 2023
Revised: 30 March 2023
Accepted: 11 May 2023
Published: 22 September 2023
Issue date: April 2024

Copyright

© The author(s) 2024.

Acknowledgements

This work was supported by the National Key Research and Development Program of China (No. 2021YFB3300503), Regional Innovation and Development Joint Fund of National Natural Science Foundation of China (No. U22A20167), and National Natural Science Foundation of China (No. 61872260).

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return