AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Tsinghua Science and Technology Article
PDF (1.2 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

False Negative Sample Detection for Graph Contrastive Learning

College of Data Science, Taiyuan University of Technology, Jinzhong 030600, China
Show Author Information

Abstract

Recently, self-supervised learning has shown great potential in Graph Neural Networks (GNNs) through contrastive learning, which aims to learn discriminative features for each node without label information. The key to graph contrastive learning is data augmentation. The anchor node regards its augmented samples as positive samples, and the rest of the samples are regarded as negative samples, some of which may be positive samples. We call these mislabeled samples as "false negative" samples, which will seriously affect the final learning effect. Since such semantically similar samples are ubiquitous in the graph, the problem of false negative samples is very significant. To address this issue, the paper proposes a novel model, False negative sample Detection for Graph Contrastive Learning (FD4GCL), which uses attribute and structure-aware to detect false negative samples. Experimental results on seven datasets show that FD4GCL outperforms the state-of-the-art baselines and even exceeds several supervised methods.

Keywords

contrastive learninggraph representation learningfalse negative sample detection

References

[1]
J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, Neural message passing for Quantum chemistry, in Proc. 34th Int. Conf. Machine Learning, Sydney, Australia, 2017, pp. 1263–1272.
Google Scholar
[2]
T. N. Kipf and M. Welling, Semi-supervised classification with graph convolutional networks, presented at 5th Int. Conf. Learning Representations, Toulon, France, 2017.
Google Scholar
[3]
Y. Li, D. Tarlow, M. Brockschmidt, and R. S. Zemel, Gated graph sequence neural networks, presented at 4th Int. Conf. Learning Representations, San Juan, Puerto Rico, 2016.
Google Scholar
[4]
P. Veličović, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, Graph attention networks, presented at 8th Int. Conf. Learning Representations, Vancouver, Canada, 2018.
Google Scholar
[5]
K. Xu, W. Hu, J. Leskovec, and S. Jegelka, How powerful are graph neural networks? presented at 7th Int. Conf. Learning Representations, New Orleans, LA, USA, 2019.
Google Scholar
[6]
H. Zeng, H. Zhou, A. Srivastava, R. Kannan, and V. K. Prasanna, GraphSAINT: Graph sampling based inductive learning method, presented at 8th Int. Conf. Learning Representations, Addis Ababa, Ethiopia, 2020.
Google Scholar
[7]
X. Su, S. Xue, F. Liu, J. Wu, J. Yang, C. Zhou, W. Hu, C. Paris, S. Nepal, D. Jin, et al., A comprehensive survey on community detection with deep learning, IEEE Trans. Neural Netw. Learn. Syst., .
Crossref Google Scholar
[8]
M. Zhang and Y. Chen, Link prediction based on graph neural networks, in Proc. 32nd Int. Conf. Neural Information Processing Systems, Montréal, Canada, 2018, pp. 5171–5181.
Google Scholar
[9]
F. Errica, M. Podda, D. Bacciu, and A. Micheli, A fair comparison of graph neural networks for graph classification, presented at 8th Int. Conf. Learning Representations, Addis Ababa, Ethiopia, 2020.
Google Scholar
[10]
F. Y. Sun, J. Hoffmann, V. Verma, and J. Tang, InfoGraph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization, presented at 8th Int. Conf. Learning Representations, Addis Ababa, Ethiopia, 2020.
Google Scholar
[11]
P. Bachman, R. D. Hjelm, and W. Buchwalter, Learning representations by maximizing mutual information across views, in Proc. 33rd Int. Conf. Neural Information Processing Systems, Vancouver, Canada, 2019, pp. 15535–15545.
Google Scholar
[12]
K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, Momentum contrast for unsupervised visual representation learning, in Proc. 2020 IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 9726–9735.
Crossref Google Scholar
[13]
Y. Tian, D. Krishnan, and P. Isola, Contrastive multiview coding, in Proc. 16th European Conf. Computer Vision – ECCV 2020, Glasgow, UK, 2020, pp. 776–794.
Crossref Google Scholar
[14]
R. Collobert and J. Weston, A unified architecture for natural language processing: Deep neural networks with multitask learning, in Proc. 25th Int. Conf. Machine learning, Helsinki, Finland, 2008, pp. 160–167.
Crossref Google Scholar
[15]
A. Mnih and K. Kavukcuoglu, Learning word embeddings efficiently with noise contrastive estimation, in Proc. 26th Int. Conf. Neural Information Processing Systems, Lake Tahoe, NV, USA, 2013, pp. 2265–2273.
Google Scholar
[16]
P. Veličkovic̀, W. Fedus, W. L. Hamilton, P. Liò, Y. Bengio, and R. D. Hjelm, Deep graph infomax, presented at 7th Int. Conf. Learning Representations, New Orleans, LA, USA, 2019.
Google Scholar
[17]
K. Hassani and A. H. Khasahmadi, Contrastive multi-view representation learning on graphs, in Proc. 37th Int. Conf. Machine Learning, Virtual Event, 2020, pp. 4116–4126.
Google Scholar
[18]
Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, and L. Wang, Deep graph contrastive representation learning, arXiv preprint arXiv: 2006.04131, 2020.
Google Scholar
[19]
Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, and L. Wang, Graph contrastive learning with adaptive augmentation, in Proc. Web Conference 2021, Ljubljana, Slovenia, 2021, pp. 2069–2080.
Crossref Google Scholar
[20]
T. S. Chen, W. C. Hung, H. Y. Tseng, S. Y. Chien, and M. H. Yang, Incremental false negative detection for contrastive learning, presented at 10th Int. Conf. Learning Representations, Virtual Event, 2022.
Google Scholar
[21]
Z. Hou, X. Liu, Y. Cen, Y. Dong, H. Yang, C. Wang, and J. Tang, GraphMAE: Self-supervised masked graph autoencoders, in Proc. 28th ACM SIGKDD Conf. Knowledge Discovery and Data Mining, Washington, DC, USA, 2022, pp. 594–604.
Crossref Google Scholar
[22]
N. Lee, J. Lee, and C. Park, Augmentation-free self-supervised learning on graphs, Proc. AAAI Conf. Artif. Intell., vol. 36, no. 7, pp. 7372–7380, 2022.
Crossref Google Scholar
[23]
X. Gong, C. Yang, and C. Shi, MA-GCL: Model augmentation tricks for graph contrastive learning, arXiv preprint arXiv: 2212.07035, 2022.
Google Scholar
[24]
J. Xia, L. Wu, G. Wang, J. Chen, and S. Z. Li, ProGCL: Rethinking hard negative mining in graph contrastive learning, presented at 39th Int. Conf. Machine Learning, Baltimore, MD, USA, 2022.
Google Scholar
[25]
C. Y. Chuang, J. Robinson, Y. C. Lin, A. Torralba, and S. Jegelka, Debiased contrastive learning, presented at 34th Conf. Neural Information Processing Systems, Vancouver, Canada, 2020.
Google Scholar
[26]
T. Huynh, S. Kornblith, M. R. Walter, M. Maire, and M. Khademi, Boosting contrastive self-supervised learning with false negative cancellation, arXiv preprint arXiv: 2011.11765, 2020.
Google Scholar
[27]
P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan, Supervised contrastive learning, presented at 34th Conf. Neural Information Processing Systems, Vancouver, Canada, 2020.
Google Scholar
[28]
X. Qin, N. Sheikh, B. Reinwald, and L. Wu, Relation-aware graph attention model with adaptive self-adversarial training, Proc. AAAI Conf. Artif. Intell., vol. 35, no. 11, pp. 9368–9376, 2021.
Crossref Google Scholar
[29]
M. Caron, P. Bojanowski, A. Joulin, and M. Douze, Deep clustering for unsupervised learning of visual features, in Proc. 15th European Conf. Computer Vision, Munich, Germany, 2018, pp. 139–156.
Crossref Google Scholar
[30]
J. Yang, D. Parikh, and D. Batra, Joint unsupervised learning of deep representations and image clusters, in Proc. 2016 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 5147–5156.
Crossref Google Scholar
[31]
J. Xie, R. Girshick, and A. Farhadi, Unsupervised deep embedding for clustering analysis, in Proc. 33rd Int. Conf.Machine Learning , New York, NY, USA, 2016, pp. 478–487.
Google Scholar
[32]
M. Caron, P. Bojanowski, J. Mairal, and A. Joulin, Unsupervised pre-training of image features on non-curated data, in Proc. 2019 IEEE/CVF Int. Conf. Computer Vision (ICCV), Seoul, Republic of Korea, 2019, pp. 2959–2968.
Crossref Google Scholar
[33]
H. Zhao, X. Yang, Z. Wang, E. Yang, and C. Deng, Graph debiased contrastive learning with joint representation clustering, in Proc. Thirtieth Int. Joint Conf. Artificial Intelligence, Montreal, Canada, 2021, pp. 3434–3440.
Crossref Google Scholar
[34]
D. Xu, W. Cheng, D. Luo, H. Chen, and X. Zhang, InfoGCL: Information-aware graph contrastive learning, presented at 35th Conf. Neural Information Processing Systems, Virtual Event, 2021.
Google Scholar
[35]
D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, presented at 3rd Int. Conf. Learning Representations, San Diego, CA, USA, 2015.
Google Scholar
Tsinghua Science and Technology
Volume 29 Issue 2,
April 2024
Pages 529-542
DOI: 10.26599/TST.2023.9010043
Cite this article:
Zhang B, Wang L. False Negative Sample Detection for Graph Contrastive Learning. Tsinghua Science and Technology, 2024, 29(2): 529-542. https://doi.org/10.26599/TST.2023.9010043

310

Views

48

Downloads

0

Crossref

0

Web of Science

0

Scopus

0

CSCD

Altmetrics
Received: 02 February 2023
Revised: 30 March 2023
Accepted: 11 May 2023
Published: 22 September 2023
© The author(s) 2024.

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
Return