Open Access
Issue
Published:
29 September 2022
Fusing Syntactic Structure Information and Lexical Semantic Information for End-to-End Aspect-Based Sentiment Analysis
Yan Yang1(
),
),
Keywords:
natural language processing, deep learning, aspect-based sentiment analysis, graph convolutional
Cite this article:
Bie Y, Yang Y, Zhang Y.
Fusing Syntactic Structure Information and Lexical Semantic Information for End-to-End Aspect-Based Sentiment Analysis.
Tsinghua Science and Technology,
2023, 28(2): 230-243.
https://doi.org/10.26599/TST.2021.9010095
Download citation
715
Views
108
Downloads
Citations
5
Crossref
2
WoS
6
Scopus
0
CSCD
The aspect-based sentiment analysis (ABSA) consists of two subtasks—aspect term extraction and aspect sentiment prediction. Most methods conduct the ABSA task by handling the subtasks in a pipeline manner, whereby problems in performance and real application emerge. In this study, we propose an end-to-end ABSA model, namely, SSi-LSi, which fuses the syntactic structure information and the lexical semantic information, to address the limitation that existing end-to-end methods do not fully exploit the text information. Through two network branches, the model extracts syntactic structure information and lexical semantic information, which integrates the part of speech, sememes, and context, respectively. Then, on the basis of an attention mechanism, the model further realizes the fusion of the syntactic structure information and the lexical semantic information to obtain higher quality ABSA results, in which way the text information is fully used. Subsequent experiments demonstrate that the SSi-LSi model has certain advantages in using different text information.
menu
Abstract
Full text
Outline
About this article
Fusing Syntactic Structure Information and Lexical Semantic Information for End-to-End Aspect-Based Sentiment Analysis
Yan Yang1(
),
),
Abstract
The aspect-based sentiment analysis (ABSA) consists of two subtasks—aspect term extraction and aspect sentiment prediction. Most methods conduct the ABSA task by handling the subtasks in a pipeline manner, whereby problems in performance and real application emerge. In this study, we propose an end-to-end ABSA model, namely, SSi-LSi, which fuses the syntactic structure information and the lexical semantic information, to address the limitation that existing end-to-end methods do not fully exploit the text information. Through two network branches, the model extracts syntactic structure information and lexical semantic information, which integrates the part of speech, sememes, and context, respectively. Then, on the basis of an attention mechanism, the model further realizes the fusion of the syntactic structure information and the lexical semantic information to obtain higher quality ABSA results, in which way the text information is fully used. Subsequent experiments demonstrate that the SSi-LSi model has certain advantages in using different text information.
Keywords:
natural language processing, deep learning, aspect-based sentiment analysis, graph convolutional
References(44)
[1]
B. Liu, Sentiment Analysis and Opinion Mining. San Rafael, CA, USA: Morgan & Claypool Publishers, 2012.
[2]
M. Bouazizi and T. Ohtsuki, Multi-class sentiment analysis on twitter: Classification performance and challenges, Big Data Mining and Analytics, vol. 2, no. 3, pp. 181–194, 2019.
[3]
F. Wang, M. Lan, and W. Wang, Towards a one-stop solution to both aspect extraction and sentiment analysis tasks with neural multi-task learning, in Proc. 2018 Int. Joint Conf. Neural Networks (IJCNN), Rio de Janeiro, Brazil, 2018, pp. 1–8.
[4]
X. Li, L. Bing, P. Li, and W. Lam, A unified model for opinion target extraction and target sentiment prediction, Proc. AAAI Conf. Artif. Intell., vol. 33, no. 1, pp. 6714–6721, 2019.
[5]
H. Luo, T. Li, B. Liu, and J. Zhang, DOER: Dual cross-shared RNN for aspect term-polarity co-extraction, in Proc. 57th Ann. Meeting of the Association for Computational Linguistics, Florence, Italy, 2019, pp. 591–601.
[6]
R. He, W. S. Lee, H. T. Ng, and D. Dahlmeier, An interactive multi-task learning network for end-to-end aspect-based sentiment analysis, in Proc. 57th Ann. Meeting of the Association for Computational Linguistics, Florence, Italy, 2019, pp. 504–515.
[7]
X. Li, L. Bing, W. Zhang, and W. Lam, Exploiting BERT for end-to-end aspect-based sentiment analysis, in Proc. 5th Workshop on Noisy User-Gernerated Text (W-NUT 2019), Hong Kong, China, 2019, pp. 34–41.
[8]
Z. Li, X. Li, Y. Wei, L. Bing, Y. Zhang, and Q. Yang, Transferable end-to-end aspect-based sentiment analysis with selective adversarial learning, in Proc. 2019 Conf. Empirical Methods in Natural Language Proc., Hong Kong, China, 2019, pp. 4590–4600.
[9]
M. Mitchell, J. Aguilar, T. Wilson, and B. Van Durme, Open domain targeted sentiment, in Proc. 2013 Conf. Empirical Methods in Natural Language Proc., Seattle, WA, USA, 2013, pp. 1643–1654.
[10]
M. S. Zhang, Y. Zhang, and D. T. Vo, Neural networks for open domain targeted sentiment, in Proc. 2015 Conf. Empirical Methods in Natural Language Proc., Lisbon, Portugal, 2015, pp. 612–621.
[11]
H. Luo, L. Ji, T. Li, D. Jiang, and N. Duan, GRACE: Gradient harmonized and cascaded labeling for aspect-based sentiment analysis, in Proc. 2020 Conf. Empirical Methods in Natural Language Proc., online, 2020, pp. 54–64.
[12]
Y. Liang, F. Meng, J. Zhang, J. Xu, Y. Chen, and J. Zhou, A dependency syntactic knowledge augmented interactive architecture for end-to-end aspect-based sentiment analysis, arXiv preprint arXiv: 2004.01951, 2020.
[13]
H. Yang, B. Zeng, J. Yang, Y. Song, and R. Xu, A multi-task learning model for Chinese-oriented aspect polarity classification and aspect term extraction, Neurocomputing, vol. 419, pp. 344–356, 2020.
[14]
F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, The graph neural network model, IEEE Trans. Neural Networks, vol. 20, no. 1, pp. 61–80, 2009.
[15]
H. Ye, Z. Yan, and Z. Luo, Dependency-tree based convolutional neural networks for aspect term extraction, in Proc. 2017 Pacific-Asia Conf. Knowledge Discovery and Data Mining, Jeju, Republic of South Korea, 2017, pp. 350–362.
[16]
H. Luo, T. Li, B. Liu, B. Wang, and H. Unger, Improving aspect term extraction with bidirectional dependency tree representation, IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 27, no. 7, pp. 1201–1212, 2019.
[17]
Y. Yin, F. Wei, L. Dong, K. Xu, M. Zhang, and M. Zhou, Unsupervised word and dependency path embeddings for aspect term extraction, in Proc. 25th Int. Joint Conf. Artificial Intelligence, New York, NY, USA, 2016, pp. 2979–2985.
[18]
K. Sun, R. Zhang, S. Mensah, Y. Mao, and X. Liu, Aspect-level sentiment analysis via convolution over dependency tree, in Proc. 2019 Conf. Empirical Methods in Natural Language Proc. and 9th Int. Joint Conf. Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, 2019, pp. 5679–5688.
[19]
M. H. Phan and P. Ogunbona, Modelling context and syntactical features for aspect-based sentiment analysis, in Proc. 58th Ann. Meeting of the Association for Computational Linguistics, online, 2020, pp. 3211–3220.
[20]
K. Wang, W. Shen, Y. Yang, X. Quan, and R. Wang, Relational graph attention network for aspect-based sentiment analysis, in Proc. 58th Ann. Meeting of the Association for Computational Linguistics, online, 2020, pp. 3229–3238.
[21]
P. Zhao, L. Hou, and O. Wu, Modeling sentiment dependencies with graph convolutional networks for aspect-level sentiment classification, Knowledge-Based Systems, vol. 193, p.105443, 2020.
[22]
T. N. Kipf and M. Welling, Semi-supervised classification with graph convolutional networks, arXiv preprint arXiv: 1609.02907, 2016.
[23]
P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, Graph attention networks, arXiv preprint arXiv: 1710.10903, 2017.
[24]
R. Xie, X. Yuan, Z. Liu, and M. Sun, Lexical sememe prediction via word embeddings and matrix factorization, in Proc. 26th Int. Joint Conf. Artificial Intelligence, Melbourne, Australia, 2017, pp. 4200–4206.
[25]
L. Luo, X. Ao, Y. Song, J. Li, X. Yang, Q. He, and D. Yu, Unsupervised Neural Aspect Extraction with Sememes, in Proc. 28th Int. Joint Conf. Artificial Intelligence, Macao, China, 2019, pp. 5123–5129.
[26]
Y. Liu, R. Xie, Z. Liu, and M. Sun, Improved word representation learning with sememes, in Proc. 55th Ann. Meeting of the Association for Computational Linguistics, Vancouver, Canada, 2017, pp. 2049–2058.
[27]
X. Zeng, C. Yang, C. Tu, Z. Liu, and M. Sun, Chinese LIWC lexicon expansion via hierarchical classification of word embeddings with sememe attention, in Proc. 32nd AAAI Conf. Artif. Intell. (AAAI-18), New Orleans, LA, USA, 2018, pp. 5650–5657.
[28]
W. Y. Wang, S. J. Pan, D. Dahlmeier, and X. Xiao, Coupled multi-layer attentions for co-extraction of aspect and opinion terms, in Proc. 31st AAAI Conference on Artificial Intelligence, San Francisco, CA, USA, 2017, pp. 3316–3322.
[29]
H. Xu, B. Liu, L. Shu, and P. S. Yu, Double embeddings and CNN-based sequence labeling for aspect extraction, in Proc. 56th Annnual Meeting of the Association for Computational Linguistics, Melbourne, Australia, 2018, pp. 592–598.
[30]
X. Li, L. Bing, P. Li, W. Lam, and Z. Yang, Aspect term extraction with history attention and selective transformation, in Proc. 27th Int. Joint Conf. Artificial Intelligence, Stockholm, Sweden, 2018, pp. 4194–4200.
[31]
Y. Q. Wang, M. L. Huang, X. Y. Zhu, and L. Zhao, Attention-based LSTM for aspect-level sentiment classification, in Proc. 2016 Conf. Empirical Methods in Natural Language Processing, Austin, TX, USA, 2016, pp. 606–615.
[32]
Z. Fan, Z. Wu, X. Dai, S. Huang, and J. Chen, Target-oriented opinion words extraction with target-fused neural sequence labeling, in Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA, 2019, pp. 2509–2518.
[33]
R. D. He, W. S. Lee, H. T. Ng, and D. Dahlmeier, Exploiting document knowledge for aspect-level sentiment classification, in Proc. 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia, 2018, pp. 579–585.
[34]
X. Li, L. D. Bing, W. Lam, and B. Shi, Transformation networks for target-oriented sentiment classification, in Proc. 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia, 2018, pp. 946–956.
[35]
R. D. He, W. S. Lee, H. T. Ng, and D. Dahlmeier, An unsupervised neural attention model for aspect extraction, in Proc. 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, Canada, 2017, pp. 388–397.
[36]
J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, in Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA, 2019, pp. 4171–4186.
[37]
J. Pennington, R. Socher, and C. D. Manning, GloVe: Global vectors for word representation, in Proc. 2014 Conf. Empirical Methods in Natural Language Processing, Doha, Qatar, 2014, pp. 1532–1543.
[38]
M. Pontiki, D. Galanis, J. Pavlopoulos, H. Papageorgiou, I. Androutsopoulos, and S. Manandhar, SemEval-2014 task 4: Aspect based sentiment analysis, in Proc. 8th Int. Workshop on Semantic Evaluation (SemEval 2014), Dublin, Ireland, 2014, pp. 27–35.
[39]
G. Lample, M. Ballesteros, S. Subramanian, K. Kawakami, and C. Dyer, Neural architectures for named entity recognition, in Proc. 2016 Conf. North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, CA, USA, 2016, pp. 260–270.
[40]
X. Z. Ma and E. Hovy, End-to-end sequence labeling via bidirectional LSTM-CNNs-CRF, in Proc. 54th Annu. Meeting of the Association for Computational Linguistics, Berlin, Germany, 2016, pp. 1064–1074.
[41]
L. Liu, J. Shang, X. Ren, F. F. Xu, H. Gui, J. Peng, and J. Han, Empower sequence labeling with task-aware neural language model, in Proc. AAAI Conf. Artif. Intell., New Orleans, LA, USA, 2018, pp. 5253–5260.
[42]
M. Gardner, J. Grus, M. Neumann, O. Tafjord, P. Dasigi, N. F. Liu, M. Peters, M. Schmitz, and L. Zettlemoyer, AllenNLP: A deep semantic natural language processing platform, in Proc. Workshop for NLP Open Source Software, Melbourne, Australia, 2018, pp. 1–6.
[43]
F. Qi, C. Yang, Z. Liu, Q. Dong, M. Sun, and Z. Dong, OpenHowNet: An open sememe-based lexical knowledge base, arXiv preprint arXiv: 1901.09957, 2019.
[44]
I. Loshchilov and F. Hutter, SGDR: Stochastic gradient descent with warm restarts, arXiv preprint arXiv: 1608.03983, 2016.
Publication history
Copyright
Acknowledgements
Rights and permissions
Publication history
Received: 15 June 2021
Revised: 29 October 2021
Accepted: 22 December 2021
Published:
29 September 2022
Issue date: April 2023
Copyright
© The author(s) 2023.
Acknowledgements
This work was supported by the National Natural Science Foundation of China (No. 61976247).
Rights and permissions
The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
FEEDBACK 
TOP