Journal Home > Volume 4 , Issue 3
Big Data Mining and Analytics 2021, 4(3): 195-207 https://doi.org/10.26599/BDMA.2021.9020003
Open Access | Issue | Published: 12 May 2021

A Multitask Multiview Neural Network for End-to-End Aspect-Based Sentiment Analysis

Show Author's Information Yong Bie1,  Yan Yang1( )
School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu 611756, China
Keywords:
natural language processing, deep learning, multitask learning, multiview learning, aspect-based sentiment analysis
Cite this article:
Bie Y, Yang Y. A Multitask Multiview Neural Network for End-to-End Aspect-Based Sentiment Analysis. Big Data Mining and Analytics, 2021, 4(3): 195-207. https://doi.org/10.26599/BDMA.2021.9020003
Download citation
EndNote(RIS)
BibTeX

1405

Views

90

Downloads

Citations

32

Crossref

28

WoS

42

Scopus

5

CSCD

Abstract Full text About this article

The aspect-based sentiment analysis (ABSA) consists of two subtasks'aspect term extraction and aspect sentiment prediction. Existing methods deal with both subtasks one by one in a pipeline manner, in which there lies some problems in performance and real application. This study investigates the end-to-end ABSA and proposes a novel multitask multiview network (MTMVN) architecture. Specifically, the architecture takes the unified ABSA as the main task with the two subtasks as auxiliary tasks. Meanwhile, the representation obtained from the branch network of the main task is regarded as the global view, whereas the representations of the two subtasks are considered two local views with different emphases. Through multitask learning, the main task can be facilitated by additional accurate aspect boundary information and sentiment polarity information. By enhancing the correlations between the views under the idea of multiview learning, the representation of the global view can be optimized to improve the overall performance of the model. The experimental results on three benchmark datasets show that the proposed method exceeds the existing pipeline methods and end-to-end methods, proving the superiority of our MTMVN architecture.


menu
Abstract
Full text
Outline
About this article

A Multitask Multiview Neural Network for End-to-End Aspect-Based Sentiment Analysis

Show Author's information Yong Bie1, Yan Yang1( )
School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu 611756, China

Abstract

The aspect-based sentiment analysis (ABSA) consists of two subtasks'aspect term extraction and aspect sentiment prediction. Existing methods deal with both subtasks one by one in a pipeline manner, in which there lies some problems in performance and real application. This study investigates the end-to-end ABSA and proposes a novel multitask multiview network (MTMVN) architecture. Specifically, the architecture takes the unified ABSA as the main task with the two subtasks as auxiliary tasks. Meanwhile, the representation obtained from the branch network of the main task is regarded as the global view, whereas the representations of the two subtasks are considered two local views with different emphases. Through multitask learning, the main task can be facilitated by additional accurate aspect boundary information and sentiment polarity information. By enhancing the correlations between the views under the idea of multiview learning, the representation of the global view can be optimized to improve the overall performance of the model. The experimental results on three benchmark datasets show that the proposed method exceeds the existing pipeline methods and end-to-end methods, proving the superiority of our MTMVN architecture.

Keywords: natural language processing, deep learning, multitask learning, multiview learning, aspect-based sentiment analysis

References(43)

[1]
M. Bouazizi, T. Ohtsuki, Multi-class sentiment analysis on twitter: Classification performance and challenges, Big Data Mining and Analytics, vol. 2, no. 3, pp. 181-194, 2019.
DOI Google Scholar
[2]
B. Liu, Sentiment Analysis and Opinion Mining. San Rafael, CA, USA: Morgan & Claypool Publishers, 2012.
[3]
P. Jiang, C. X. Zhang, H. P. Fu, Z. D. Niu, and Q. Yang, An approach based on tree kernels for opinion mining of online product reviews, in 2010 IEEE Int. Conf. Data Mining, Sydney, Australia, 2010, pp. 256-265.
DOI
[4]
W. Jin and H. H. Ho, A novel lexicalized hmm-based learning framework for web opinion mining, in Proc. 26th Int. Conf. Machine Learning, Montreal, Canada, 2009, pp. 465-472.
DOI
[5]
N. Jakob and I. Gurevych, Extracting opinion targets in a single- and cross-domain setting with conditional random fields, in Proc. 2010 Conf. Empirical Methods in Natural Language Proc., Boston, MA, USA, 2010, pp. 1035-1045.
[6]
F. T. Li, C. Han, M. L. Huang, X. Y. Zhu, Y. J. Xia, S. Zhang, and H. Yu, Structure-aware review mining and summarization, in Proc. 23rd Int. Conf. Computational Linguistics, Beijing, China, 2010, pp. 653-661.
[7]
S. Poria, E. Cambria, and A. Gelbukh, Aspect extraction for opinion mining with a deep convolutional neural network, Knowl. Based Syst., vol. 108, pp. 42-49, 2016.
DOI Google Scholar
[8]
W. Y. Wang, S. J. Pan, D. Dahlmeier, and X. K. Xiao, Coupled multi-layer attentions for co-extraction of aspect and opinion terms, in Proc. 31st AAAI Conf. Artificial Intelligence, San Francisco, CA, USA, 2017, pp. 3316-3322.
[9]
H. Ye, Z. C. Yan, Z. C. Luo, and W. H. Chao, Dependency-tree based convolutional neural networks for aspect term extraction, in Pacific-Asia Conf. Knowledge Discovery and Data Mining, H. Ye, Z. Yan, Z. Luo, and W. Chao, Eds. Cham, Germany: Springer, 2017, pp. 350-362.
DOI
[10]
H. S. Luo, T. R. Li, B. Liu, B. Wang, and H. Unger, Improving aspect term extraction with bidirectional dependency tree representation, IEEE/ACM Trans. Audio Speech Language Proc., vol. 27, no. 7, pp. 1201-1212, 2019.
DOI Google Scholar
[11]
H. Xu, B. Liu, L. Shu, and P. S. Yu, Double embeddings and CNN-based sequence labeling for aspect extraction, in Proc. 56th Ann. Meeting of the Association for Computational Linguistics, Melbourne, Australia, 2018, pp. 592-598.
DOI
[12]
Y. Q. Wang, M. L. Huang, X. Y. Zhu, and L. Zhao, Attention-based LSTM for aspect-level sentiment classification, in Proc. 2016 Conf. Empirical Methods in Natural Language Proc., Austin, TX, USA, 2016, pp. 606-615.
DOI
[13]
D. Y. Tang, B. Qin, and T. Liu, Aspect level sentiment classification with deep memory network, in Proc. 2016 Conf. Empirical Methods in Natural Language Proc., Austin, TX, USA, 2016, pp. 214-224.
DOI
[14]
F. F. Fan, Y. S. Feng, and D. Y. Zhao, Multi-grained attention network for aspect-Level sentiment classification, in Proc. 2018 Conf. Empirical Methods in Natural Language Proc., Brussels, Belgium, 2018, pp. 3433-3442.
DOI
[15]
P. Chen, Z. Q. Sun, L. D. Bing, and W. Yang, Recurrent attention network on memory for aspect sentiment analysis, in Proc. 2017 Conf. Empirical Methods in Natural Language Proc., Copenhagen, Denmark, 2017, pp. 452-461.
DOI
[16]
D. H. Ma, S. J. Li, X. D. Zhang, and H. F. Wang, Interactive attention networks for aspect-level sentiment classification, in Proc. 26th Int. Joint Conf. Artificial Intelligence, Melbourne, Australia, 2017, pp. 4068-4074.
DOI
[17]
B. X. Huang, Y. L. Ou, and K. M. Carley, Aspect level sentiment classification with attention-over-attention neural networks, in Int. Conf. Social Computing, Behavioral-Cultural Modeling & Prediction and Behavior Representation in Modeling and Simulation (SBP-BRiMS), R. Thomson, C. Dancy, A. Hyder, and H. Bisgin, Eds. Cham, Germany: Springer, 2018, pp. 197-206.
DOI
[18]
L. S. Li, Y. Liu, and A. Q. Zhou, Hierarchical attention based position-aware network for aspect-level sentiment analysis, in Proc. 22nd Conf. Computational Natural Language Learning, Brussels, Belgium, 2018, pp. 181-189.
DOI
[19]
Y. W. Song, J. H. Wang, T. Jiang, Z. Y. Liu, and Y. H. Rao, Attentional encoder network for targeted sentiment classification, arXiv preprint arXiv: 1902.09314v2, 2019.
Google Scholar
[20]
X. Li, L. D. Bing, W. Lam, and B. Shi, Transformation networks for target-oriented sentiment classification, in Proc. 56th Ann. Meeting of the Association for Computational Linguistics, Melbourne, Australia, 2018, pp. 946-956.
DOI
[21]
F. X. Wang, M. Lan, and W. T. Wang, Towards a one-stop solution to both aspect extraction and sentiment analysis tasks with neural multi-task learning, in Proc. 2018 Int. Joint Conf. Neural Networks (IJCNN), Rio de Janeiro, Brazil, 2018, pp. 1-8.
DOI
[22]
X. Li, L. D. Bing, P. J. Li, and W. Lam, A unified model for opinion target extraction and target sentiment prediction, Proc. AAAI Conf. Artif. Intell., vol. 33, no. 1, pp. 6714-6721, 2019.
DOI Google Scholar
[23]
R. D. He, W. S. Lee, H. T. Ng, and D. Dahlmeier, An interactive multi-task learning network for end-to-end aspect-based sentiment analysis, in Proc. 57th Ann. Meeting of the Association for Computational Linguistics, Florence, Italy, 2019, pp. 504-515.
DOI
[24]
H. S. Luo, T. R. Li, B. Liu, and J. B. Zhang, DOER: Dual cross-shared RNN for aspect term-polarity co-extraction, in Proc. 57th Ann. Meeting of the Association for Computational Linguistics, Florence, Italy, 2019, pp. 591-601.
DOI
[25]
Y. Kim, Convolutional neural networks for sentence classification, in Proc. 2014 Conf. Empirical Methods in Natural Language Proc., Doha, Qatar, 2014, pp. 1746-1751.
DOI
[26]
K. Cho, B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, Learning phrase representations using RNN encoder-decoder for statistical machine translation, in Proc. 2014 Conf. Empirical Methods in Natural Language Processing, Doha, Qatar, 2014, pp. 1724-1734.
DOI
[27]
L. Sha, X. D. Zhang, F. Qian, B. B. Chang, and Z. F. Sui, A Multi-view fusion neural network for answer selection, in 32nd AAAI Conf. Artificial Intelligence, New Orleans, LA, USA, 2018, pp. 5422-5429.
[28]
G. Andrew, R. Arora, J. Bilmes, and K. Livescu, Deep canonical correlation analysis, in Proc. 30th Int. Conf. Machine Learning, Atlanta, GA, USA, 2013, pp. 1247-1255.
[29]
S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural Comput., vol. 9, no. 8, pp. 1735-1780, 1997.
DOI Google Scholar
[30]
J. Weston, S. Chopra, and A. Bordes, Memory networks, arXiv preprint arXiv: 1410.3916, 2014.
Google Scholar
[31]
Y. M. Cui, Z. P. Chen, S. Wei, S. J. Wang, T. Liu, and G. P. Hu, Attention-over-attention neural networks for reading comprehension, in Proc. 55th Ann. Meeting of the Association for Computational Linguistics, Vancouver, Canada, 2017, pp. 593-602.
DOI
[32]
J. Pennington, R. Socher, and C. D. Manning, Glove: Global vectors for word representation, in Proc. 2014 Conf. Empirical Methods in Natural Language Proc., Doha, Qatar, 2014, pp. 1532-1543.
DOI
[33]
K. M. He, X. Y. Zhang, S. Q. Ren, and J. Sun, Deep residual learning for image recognition, in Proc. 2016 IEEE Conf. Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016, pp. 770-778.
DOI
[34]
M. Pontiki, D. Galanis, J. Pavlopoulos, H. Papageorgiou, I. Androutsopoulos, and S. Manandhar, Semeval-2014 task 4: Aspect based sentiment analysis, in Proc. 8th Int. Workshop on Semantic Evaluation (SemEval 2014), Dublin, Ireland, 2014, pp. 27-35.
DOI
[35]
M. Mitchell, J. Aguilar, T. Wilson, and B. Van Durme, Open domain targeted sentiment, in Proc. 2013 Conf. Empirical Methods in Natural Language Proc., Seattle, WA, USA, 2013, pp. 1643-1654.
[36]
M. S. Zhang, Y. Zhang, and D. T. Vo, Neural networks for open domain targeted sentiment, in Proc. 2015 Conf. Empirical Methods in Natural Language Proc., Lisbon, Portugal, 2015, pp. 612-621.
DOI
[37]
H. Li and W. Lu, Learning latent sentiment scopes for entity-level sentiment analysis, in Proc. 31st AAAI Conf. Artificial Intelligence, San Francisco, CA, USA, 2017, pp. 3482-3489.
[38]
G. Lample, M. Ballesteros, S. Subramanian, K. Kawakami, and C. Dyer, Neural architectures for named entity recognition, in Proc. 2016 Conf. North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, CA, USA, 2016, pp. 260-270.
DOI
[39]
X. Z. Ma and E. Hovy, End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF, in Proc. 54th Annu. Meeting of the Association for Computational Linguistics, Berlin, Germany, 2016, pp. 1064-1074.
DOI
[40]
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł Kaiser, and I. Polosukhin, Attention is all you need, in Proc. 31st Conf. Neural Information Proc. Systems, Long Beach, CA, USA, 2017, pp. 5998-6008.
[41]
X. Glorot and Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, in Proc. 13th Int. Conf. Artificial Intelligence and Statistics, Sardinia, Italy, vol. 9, pp. 249-256.
[42]
K. M. He, X. Y. Zhang, S. Q. Ren, and J. Sun, Delving deep into rectifiers: surpassing human-level performance on ImageNet classification, in 2015 IEEE Int. Conf. Computer Vision (ICCV), Santiago, Chile, 2015, pp. 1026-1034.
DOI
[43]
F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, The graph neural network model, IEEE Trans. Neural Networks, vol. 20, no. 1, pp. 61-80, 2009.
DOI Google Scholar
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 03 January 2021
Accepted: 25 January 2021
Published: 12 May 2021
Issue date: September 2021

Copyright

© The author(s) 2021

Acknowledgements

This work was supported by the National Natural Science Foundation of China (No. 61976247).

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return