Open Access
Issue
Published:
13 November 2021
Adversarial Training for Supervised Relation Extraction
Yanhua Yu(
),
),
Keywords:
generative adversarial network, relation extraction, piecewise convolution neural network, adversarial training
Cite this article:
Yu Y, He K, Li J.
Adversarial Training for Supervised Relation Extraction.
Tsinghua Science and Technology,
2022, 27(3): 610-618.
https://doi.org/10.26599/TST.2020.9010059
Download citation
473
Views
56
Downloads
Citations
2
Crossref
3
WoS
3
Scopus
1
CSCD
Most supervised methods for relation extraction (RE) involve time-consuming human annotation. Distant supervision for RE is an efficient method to obtain large corpora that contains thousands of instances and various relations. However, the existing approaches rely heavily on knowledge bases (e.g., Freebase), thereby introducing data noise. Various relations and noisy labeling instances make the issue difficult to solve. In this study, we propose a model based on a piecewise convolution neural network with adversarial training. Inspired by generative adversarial networks, we adopt a heuristic algorithm to identify noisy datasets and apply adversarial training to RE. Experiments on the extended dataset of SemEval-2010 Task 8 show that our model can obtain more accurate training data for RE and significantly outperforms several competitive baseline models. Our model has an
menu
Abstract
Full text
Outline
About this article
Adversarial Training for Supervised Relation Extraction
Yanhua Yu(
),
),
Abstract
Most supervised methods for relation extraction (RE) involve time-consuming human annotation. Distant supervision for RE is an efficient method to obtain large corpora that contains thousands of instances and various relations. However, the existing approaches rely heavily on knowledge bases (e.g., Freebase), thereby introducing data noise. Various relations and noisy labeling instances make the issue difficult to solve. In this study, we propose a model based on a piecewise convolution neural network with adversarial training. Inspired by generative adversarial networks, we adopt a heuristic algorithm to identify noisy datasets and apply adversarial training to RE. Experiments on the extended dataset of SemEval-2010 Task 8 show that our model can obtain more accurate training data for RE and significantly outperforms several competitive baseline models. Our model has an
Keywords:
generative adversarial network, relation extraction, piecewise convolution neural network, adversarial training
References(35)
[1]
A. Fader, S. Soderland, and O. Etzioni, Identifying relations for open information extraction, in Proc. Conf. Empirical Methods in Natural Language Proc., Stroudsburg, PA, USA, 2011, pp. 1535-1545.
[2]
K. Xu, S. Reddy, Y. S. Feng, S. F. Huang, and D. Y. Zhao, Question answering on freebase via relation extraction and textual evidence, in Proc. 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany, 2016, pp. 2326-2336.
[3]
K. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor, Freebase: A collaboratively created graph database for structuring human knowledge, in Proc. 2008 ACM SIGMOD Int. Conf. Management of Data, Vancouver, Canada, 2008, pp. 1247-1250.
[4]
I. Hendrickx, S. N. Kim, Z. Kozareva, P. Nakov, D. Ó Séaghdha, S. Padó, M. Pennacchiotti, L. Romano, and S. Szpakowicz, SemEval-2010 Task 8: Multi-way classification of semantic relations between pairs of nominals, in Proc. 5th Int. Workshop on Semantic Evaluation, Uppsala, Sweden, 2010, pp. 33-38.
[5]
A. Mitchell, S. Strassel, S. D. Huang, and R. Zakhary, ACE 2004 Multilingual Training Corpus. Philadelphia, PA, USA: Linguistic Data Consortium, 2005.
[6]
M. Mintz, S. Bills, R. Snow, and D. Jurafsky, Distant supervision for relation extraction without labeled data, in Proc. Joint Conf. 47th Annual Meeting of the ACL and the 4th Int. Joint Conf. Natural Language Processing of the AFNLP, Stroudsburg, PA, USA, 2009, pp. 1003-1011.
[7]
S. Riedel, L. M. Yao, and A. McCallum, Modeling relations and their mentions without labeled text, in Proc. 2010 European Conf. Machine Learning and Knowledge Discovery in Databases: Part III, Berlin, Germany, 2010, pp. 148-163.
[8]
R. Hoffmann, C. L. Zhang, X. Ling, L. Zettlemoyer, and D. S. Weld, Knowledge-based weak supervision for information extraction of overlapping relations, in Proc. 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, OR, USA, 2011, pp. 541-550.
[9]
D. J. Zeng, K. Liu, Y. B. Chen, and J. Zhao, Distant supervision for relation extraction via piecewise convolutional neural networks, in Proc. 2015 Conf. Empirical Methods in Natural Language Processing, Lisbon, Portugal, 2015, pp. 1753-1762.
[10]
W. Y. Zeng, Y. K. Lin, Z. Y. Liu, and M. S. Sun, Incorporating relation paths in neural relation extraction, in Proc. 2017 Conf. Empirical Methods in Natural Language Processing, Copenhagen, Denmark, 2017, pp. 1768-1777.
[11]
C. dos Santos, B. Xiang, and B. W. Zhou, Classifying relations by ranking with convolutional neural networks, in Proc. 53rd Annual Meeting of the Association for Computational Linguistics and the 7th Int. Joint Conf. Natural Language Processing, Beijing, China, 2015, pp. 626-634.
[12]
Y. K. Lin, S. Q. Shen, Z. Y. Liu, H. B. Luan, and M. S. Sun, Neural relation extraction with selective attention over instances, in Proc. 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany, 2016, pp. 2124-2133.
[13]
P. Zhou, J. M. Xu, Z. Y. Qi, H. Y. Bao, Z. N. Chen, and B. Xu, Distant supervision for relation extraction with hierarchical selective attention, Neural Networks, vol. 108, pp. 240-247, 2018.
[14]
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, Generative adversarial nets, in Proc. 27th Int. Conf. Neural Information Processing Systems - Volume 2, Cambridge, MA, USA, 2014, pp. 2672-2680.
[15]
X. Wu, K. Xu, and P. Hall, A survey of image synthesis and editing with generative adversarial networks, Tsinghua Science and Technology, vol. 22, no. 6, pp. 660-674, 2017.
[16]
R. Socher, B. Huval, C. D. Manning, and A. Y. Ng, Semantic compositionality through recursive matrix-vector spaces, in Proc. 2012 Joint Conf. Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Stroudsburg, PA, USA, 2012, pp. 1201-1211.
[17]
D. J. Zeng, K. Liu, S. W. Lai, G. Y. Zhou, and J. Zhao, Relation classification via convolutional deep neural network, in Proc. 25th Int. Conf. Computational Linguistics: Technical Papers, Dublin, Ireland, 2014, pp. 2335-2344.
[18]
T. Y. Liu, K. X. Wang, B. B. Chang, and Z. F. Sui, A soft-label method for noise-tolerant distantly supervised relation extraction, in Proc. 2017 Conf. Empirical Methods in Natural Language Processing, Copenhagen, Denmark, 2017, pp. 1790-1795.
[19]
Y. J. Yuan, L. Y. Liu, S. L. Tang, Z. F. Zhang, Y. T. Zhuang, S. L. Pu, F. Wu, and X. Ren, Cross-relation cross-bag attention for distantly-supervised relation extraction, Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 1, pp. 419-426, 2019.
[20]
Z. X. Ye and Z. H. Ling, Distant supervision relation extraction with intra-bag and inter-bag attentions, in Proc. 2019 Conf. North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, Minnesota, 2019, pp. 2810-2819.
[21]
S. C Wu and Y. F. He, Enriching pre-trained language model with entity information for relation classification, in Proc. 28th ACM Int. Conf. Information and Knowledge Management, Beijing, China, 2019, pp. 2361-2364.
[22]
X. Han, and Z. Y. Liu, and M. S. Sun, Neural knowledge acquisition via mutual attention between knowledge graph and text, in Proc. 32nd AAAI Conf. Artificial Intelligence, New Orleans, LA, USA, 2018, pp. 4832-4839.
[23]
G. L. Ji, K. Liu, S. Z. He, and J. Zhao, Distant supervision for relation extraction with sentence-level attention and entity descriptions, in Proc. 31st AAAI Conf. Artificial Intelligence, San Francisco, CA, USA, 2017, pp. 3060-3066.
[24]
J. Feng, M. L. Huang, L. Zhao, Y. Yang, and X. Y. Zhu, Reinforcement learning for relation classification from noisy data, in Proc. 32nd AAAI Conf. Artificial Intelligence, New Orleans, LA, USA, 2018.
[25]
T. Miyato, A. M. Dai, and I. Goodfellow, Adversarial training methods for semi-supervised text classification, arXiv preprint arXiv:1605.07725, 2017.
[26]
Y. Wu, D. Bamman, and S. Russell, Adversarial training for relation extraction, in Proc. 2017 Conf. Empirical Methods in Natural Language Processing, Copenhagen, Denmark, 2017, pp. 1778-1783.
[27]
P. D. Qin, W. R. Xu, and W. Y. Wang, DSGAN: Generative adversarial training for distant supervision relation extraction, in Proc. 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia, 2018, pp. 496-505.
[28]
D. J. Im, C. D. Kim, H. Jiang, and R. Memisevic, Generating images with recurrent adversarial networks, arXiv preprint arXiv:1602.05110, 2016.
[29]
X. Z. Wang, X. Han, Z. Y. Liu, M. S. Sun, and P. Li, Adversarial training for weakly supervised event detection, in Proc. 2019 Conf. North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, Minnesota, 2019, pp. 998-1008.
[30]
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, Dropout: A simple way to prevent neural networks from overfitting, Journal of Machine Learning Research, vol. 15, no. 1, pp. 1929-1958, 2014.
[31]
J. Pennington, R. Socher, and C. D. Manning, GloVe: Global vectors for word representation, in Proc. 2014 Conf. Empirical Methods in Natural Language Processing, Doha, Qatar, 2014, pp. 1532-1543.
[32]
B. Rink and S. Harabagiu, UTD: Classifying semantic relations by combining lexical and semantic resources, in Proc. 5th Int. Workshop on Semantic Evaluation, Uppsala, Sweden, 2010, pp. 256-259.
[33]
C. dos Santos, B. Xiang, and B. W. Zhou, , Classifying relations by ranking with convolutional neural networks, in Proc. 53rd Annual Meeting of the Association for Computational Linguistics and the 7th Int. Joint Conf. Natural Language Processing, Beijing, China, 2015, pp. 626-634.
[34]
Y. T. Shen and X. J. Huang, Attention-based convolutional neural network for semantic relation extraction, in Proc. 26th Int. Conf. Computational Linguistics: Technical Papers, Osaka, Japan, 2016, pp. 2526-2536.
[35]
J. Lee, S. Seo, and Y. S. Choi, Semantic relation classification via bidirectional LSTM networks with entity-aware attention using latent entity typing, arXiv preprint arXiv:1901.08163, 2019.
Publication history
Copyright
Acknowledgements
Rights and permissions
Publication history
Received: 26 November 2020
Accepted: 09 December 2020
Published:
13 November 2021
Issue date: June 2022
Copyright
© The author(s) 2022
Acknowledgements
The research was supported in part by the National Natural Science Foundation of China (Nos. U1936104 and 2020-JCJQ-ZD-012).
Rights and permissions
The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
FEEDBACK 
TOP