Journal Home > Volume 25 , Issue 1

Uncertainty identification is an important semantic processing task. It is crucial to the quality of information in terms of factuality in many applications, such as topic detection and question answering. Factuality has become a premier concern especially in social media, in which texts are written informally. However, existing approaches that rely on lexical cues suffer greatly from the casual or word-of-mouth peculiarity of social media, in which the cue phrases are often expressed in substandard form or even omitted from sentences. To tackle these problems, this paper proposes an Attention-based Neural Framework for Uncertainty identification on social media texts, named ANFU. ANFU incorporates attention-based Long Short-Term Memory (LSTM) networks to represent the semantics of words and Convolutional Neural Networks (CNNs) to capture the most important semantics. Experiments were conducted on four datasets, including 2 English benchmark datasets used in the CoNLL-2010 task of uncertainty identification and 2 Chinese datasets of Weibo and Chinese news texts. Experimental results showed that our proposed ANFU approach outperformed the-state-of-the-art on all the datasets in terms of F1 measure. More importantly, 41.37% and 13.10% improvements were achieved over the baselines on English and Chinese social media datasets, respectively, showing the particular effectiveness of ANFU on social media texts.


menu
Abstract
Full text
Outline
About this article

An Attention-Based Neural Framework for Uncertainty Identification on Social Media Texts

Show Author's information Xu HanBinyang Li( )Zhuoran Wang
International Science and Technology Cooperation Base of Electronic System Reliability and Mathematical Interdisciplinary, Information Engineering College, Capital Normal University, Beijing 100048, China.
School of Information Science and Technology, University of International Relations, Beijing 100091, China.
Tricorn (Beijing) Technology Co. Ltd, Beijing 100029, China.

Abstract

Uncertainty identification is an important semantic processing task. It is crucial to the quality of information in terms of factuality in many applications, such as topic detection and question answering. Factuality has become a premier concern especially in social media, in which texts are written informally. However, existing approaches that rely on lexical cues suffer greatly from the casual or word-of-mouth peculiarity of social media, in which the cue phrases are often expressed in substandard form or even omitted from sentences. To tackle these problems, this paper proposes an Attention-based Neural Framework for Uncertainty identification on social media texts, named ANFU. ANFU incorporates attention-based Long Short-Term Memory (LSTM) networks to represent the semantics of words and Convolutional Neural Networks (CNNs) to capture the most important semantics. Experiments were conducted on four datasets, including 2 English benchmark datasets used in the CoNLL-2010 task of uncertainty identification and 2 Chinese datasets of Weibo and Chinese news texts. Experimental results showed that our proposed ANFU approach outperformed the-state-of-the-art on all the datasets in terms of F1 measure. More importantly, 41.37% and 13.10% improvements were achieved over the baselines on English and Chinese social media datasets, respectively, showing the particular effectiveness of ANFU on social media texts.

Keywords: attention, neural networks, social media, uncertainty identification

References(28)

[1]
G. Szarvas, V. Vincze, R. Farkas, G. Mora, and I. Gurevych, Cross-genre and cross-domain detection of se-mantic uncertainty, Computational Linguistics, vol. 38, no. 2, pp. 335-367, 2012
[2]
X. J. Li, W. Gao, and J. W. Shavlik, Detecting semantic uncertainty by learning hedge cues in sentences using an HMM, in Proceeding of the Special Interest Group on Information Retrieval (SIGIR’14), Gold Coast, AUS, 2014, pp. 89-107.
DOI
[3]
R. Farkas, V. Vincze, G. Mora, J. Csirik, and G. Szarvas, The CoNLL-2010 shared task: Learning to detect hedges and their scope in natural language text, in Proceedings of the 14th Conference on Computational Natural Language Learning-Shared Task (CoNLL’10), Uppsala, Sweden, 2010, pp. 1-12.
[4]
Y. T. Wei, X. G. You, and H. Li, Multiscale patch-based contrast measure for small infrared target detection, Pattern Recognition, vol. 58, no. 1, pp. 216-226, 2016.
[5]
Z. Y. Wei, J. W. Chen, W. Gao, B. Y. Li, and L. J. Zhou, An empirical study on uncertainty identification in social media context, in Proceedings of the Association for Computational Linguistics (ACL’13), Minneapolis, MN, USA, 2013, pp. 58-62.
[6]
D. Bahdanau, K. Cho, and Y. Bengio, Neural machine translation by jointly learning to align and translate, arXiv preprint, arXiv: 1409.0473, 2014.
[7]
Y. Kim, Convolutional neural networks for sentence classification, in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, (EMNLP’14), Doha, State of Qatar, 2014, pp. 1746-1751.
DOI
[8]
R. Morante and C. Sporleder, Modality and negation: An introduction to the special issue, Computational Linguistics, vol. 38, no. 2, pp. 223-260, 2012.
[9]
M. Light, X. Y. Qiu, and P. Srinivasan, The language of bioscience: Facts, speculations, and statements in between, in Proceedings of the BioLink 2004 Workshop on Linking Biological Literature, Ontologies and Databases: Tools for Users, Boston, MA, USA, 2004, pp. 17-24.
[10]
W. W. Chapman, D. Chu, and J. N. Dowling, Context: An algorithm for identifying contextual features from clinical text, in Proceedings of the ACL Workshop on BioNLP, Prague, Czechoslovakia, 2007, pp. 81-88.
DOI
[11]
B. Medlock, Exploring hedge identification in biomedical literature, Journal of Biomedical Informatics, vol. 41, no. 4, pp. 636-654, 2008.
[12]
E. Fernandes, C. Crestana, and R. Milidiu, Hedge detection using the RelHunter approach, in Proceedings of the 14th Conference on Computational Natural Language Learning-Shared Task (CoNLL’10), Uppsala, Sweden, 2010, pp. 64-69.
[13]
X. X. Li, J. P. Shen, X. Gao, and X. Wang, Exploiting rich features for detecting hedges and their scope, in Proceedings of the 14th Conference on Computational Natural Language Learning-Shared Task (CoNLL’10), Uppsala, Sweden, 2010, pp. 78-83.
[14]
B. Z. Tang, X. L. Wang, X. Wang, B. Yuan, and S. X. Fan, A cascade method for detecting hedges and their scope in natural language text, in Proceedings of the 14th Conference on Computational Natural Language Learning-Shared Task, (CoNLL’10), Uppsala, Sweden, 2010, pp. 13-17.
[15]
S. D. Zhang, H. Zhao, G. D. Zhou, and B. L. Lu, Hedge detection and scope finding by sequence labeling with normalized feature selection, in Proceedings of the 14th Conference on Computational Natural Language Learning-Shared Task, (CoNLL’10), Uppsala, Sweden, 2010, pp. 92-99.
[16]
E. Velldal, Detecting uncertainty in bio-medical literature: A simple disambiguation approach using sparse random indexing, in Proceedings of the International Symposium on Semantic Mining in Biomedicine (SMBM’10), Cambridge, UK, 2010, pp. 75-83.
[17]
F. Ji, X. P. Qiu, and X. J. Huang, A research on Chinese uncertainty sentence recognition, in Proceedings of the 16th China Conference on Information Retrieval (CCIR’10), Harbin, China, 2010, pp. 595-601.
[18]
V. Vincze, Uncertainty detection in natural language texts, PhD dissertation, Stanford University, Palo Alto, CA, USA, 2015.
[19]
V. Vincze, Detecting uncertainty cues in Hungarian social media texts, in Proceedings of the Workshop on Extra-Propositional Aspects of Meaning in Computational Linguistics, Osaka, Japan, 2016, pp. 11-21.
[20]
Y. Zhang and B. C. Wallace, A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification, arXiv preprint, arXiv: 1510.03820, 2015.
[21]
Z. C. Yang, D. Y. Yang, C. Dyer, X. D. He, A. Smola, and E. Hovy, Hierarchical attention networks for document classification, in Proceeding of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL’16), San Diego, CA, USA, 2016, pp. 1480-1489.
DOI
[22]
H. Adel and H. Schutze, Exploring different dimensions of attention for uncertainty detection, in Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (ECACL’17), Valencia, Spain, 2017, pp. 22-34.
DOI
[23]
S. Hochreiter and J. Schmidhuber, Flat minima, Neural Computation, vol. 9, no. 1, pp. 1-42, 1997.
[24]
M. Sundermeyer, R. Schlüter, and H. Ney, LSTM neural networks for language modeling, in Proceedings of the Interspeech, Portland, OR, USA, 2012, pp. 194-197.
DOI
[25]
K. S. Yao, T. Cohn, K. Vylomova, K. Duh, and C. Dyer, Depth-gated recurrent neural networks, arXiv preprint, arXiv: 1508.03790, 2015.
[26]
A. Graves, Generating sequences with re-current neural networks, arXiv preprint, arXiv: 1308.0850, 2013.
[27]
R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and P. Kuksa, Natural language processing (almost) from scratch, Journal of Machine Learning Research, vol. 12, no. 1, pp. 2493-2537, 2011.
[28]
T. Mikolov, K. Chen, G. Corrado, and J. Dean, Efficient estimation of word representa-tions in vector space, arXiv preprint, arXiv: 1301.3781, 2013.
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 29 July 2018
Revised: 31 December 2018
Accepted: 04 March 2019
Published: 22 July 2019
Issue date: February 2020

Copyright

© The author(s) 2020

Acknowledgements

This work was partially supported by the National Natural Science Foundation of China (Nos. 61502115, 61602326, U1636103, U1536207, and 61672361), the Fundamental Research Fund for the Central Universities (No. 3262019T29), and the Joint Funding for Capital Universities (No. SKX182010023).

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return