Journal Home > Volume 4 , issue 3

The aspect-based sentiment analysis (ABSA) consists of two subtasks'aspect term extraction and aspect sentiment prediction. Existing methods deal with both subtasks one by one in a pipeline manner, in which there lies some problems in performance and real application. This study investigates the end-to-end ABSA and proposes a novel multitask multiview network (MTMVN) architecture. Specifically, the architecture takes the unified ABSA as the main task with the two subtasks as auxiliary tasks. Meanwhile, the representation obtained from the branch network of the main task is regarded as the global view, whereas the representations of the two subtasks are considered two local views with different emphases. Through multitask learning, the main task can be facilitated by additional accurate aspect boundary information and sentiment polarity information. By enhancing the correlations between the views under the idea of multiview learning, the representation of the global view can be optimized to improve the overall performance of the model. The experimental results on three benchmark datasets show that the proposed method exceeds the existing pipeline methods and end-to-end methods, proving the superiority of our MTMVN architecture.


menu
Abstract
Full text
Outline
About this article

A Multitask Multiview Neural Network for End-to-End Aspect-Based Sentiment Analysis

Show Author's information Yong Bie1Yan Yang1( )
School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu 611756, China

Abstract

The aspect-based sentiment analysis (ABSA) consists of two subtasks'aspect term extraction and aspect sentiment prediction. Existing methods deal with both subtasks one by one in a pipeline manner, in which there lies some problems in performance and real application. This study investigates the end-to-end ABSA and proposes a novel multitask multiview network (MTMVN) architecture. Specifically, the architecture takes the unified ABSA as the main task with the two subtasks as auxiliary tasks. Meanwhile, the representation obtained from the branch network of the main task is regarded as the global view, whereas the representations of the two subtasks are considered two local views with different emphases. Through multitask learning, the main task can be facilitated by additional accurate aspect boundary information and sentiment polarity information. By enhancing the correlations between the views under the idea of multiview learning, the representation of the global view can be optimized to improve the overall performance of the model. The experimental results on three benchmark datasets show that the proposed method exceeds the existing pipeline methods and end-to-end methods, proving the superiority of our MTMVN architecture.

Keywords:

deep learning, multitask learning, multiview learning, natural language processing, aspect-based sentiment analysis
Received: 03 January 2021 Accepted: 25 January 2021 Published: 12 May 2021 Issue date: September 2021
References(43)
[1]
M. Bouazizi, T. Ohtsuki, Multi-class sentiment analysis on twitter: Classification performance and challenges, Big Data Mining and Analytics, vol. 2, no. 3, pp. 181-194, 2019.
[2]
B. Liu, Sentiment Analysis and Opinion Mining. San Rafael, CA, USA: Morgan & Claypool Publishers, 2012.
[3]
P. Jiang, C. X. Zhang, H. P. Fu, Z. D. Niu, and Q. Yang, An approach based on tree kernels for opinion mining of online product reviews, in 2010 IEEE Int. Conf. Data Mining, Sydney, Australia, 2010, pp. 256-265.
[4]
W. Jin and H. H. Ho, A novel lexicalized hmm-based learning framework for web opinion mining, in Proc. 26th Int. Conf. Machine Learning, Montreal, Canada, 2009, pp. 465-472.
[5]
N. Jakob and I. Gurevych, Extracting opinion targets in a single- and cross-domain setting with conditional random fields, in Proc. 2010 Conf. Empirical Methods in Natural Language Proc., Boston, MA, USA, 2010, pp. 1035-1045.
[6]
F. T. Li, C. Han, M. L. Huang, X. Y. Zhu, Y. J. Xia, S. Zhang, and H. Yu, Structure-aware review mining and summarization, in Proc. 23rd Int. Conf. Computational Linguistics, Beijing, China, 2010, pp. 653-661.
[7]
S. Poria, E. Cambria, and A. Gelbukh, Aspect extraction for opinion mining with a deep convolutional neural network, Knowl. Based Syst., vol. 108, pp. 42-49, 2016.
[8]
W. Y. Wang, S. J. Pan, D. Dahlmeier, and X. K. Xiao, Coupled multi-layer attentions for co-extraction of aspect and opinion terms, in Proc. 31st AAAI Conf. Artificial Intelligence, San Francisco, CA, USA, 2017, pp. 3316-3322.
[9]
H. Ye, Z. C. Yan, Z. C. Luo, and W. H. Chao, Dependency-tree based convolutional neural networks for aspect term extraction, in Pacific-Asia Conf. Knowledge Discovery and Data Mining, H. Ye, Z. Yan, Z. Luo, and W. Chao, Eds. Cham, Germany: Springer, 2017, pp. 350-362.
[10]
H. S. Luo, T. R. Li, B. Liu, B. Wang, and H. Unger, Improving aspect term extraction with bidirectional dependency tree representation, IEEE/ACM Trans. Audio Speech Language Proc., vol. 27, no. 7, pp. 1201-1212, 2019.
[11]
H. Xu, B. Liu, L. Shu, and P. S. Yu, Double embeddings and CNN-based sequence labeling for aspect extraction, in Proc. 56th Ann. Meeting of the Association for Computational Linguistics, Melbourne, Australia, 2018, pp. 592-598.
[12]
Y. Q. Wang, M. L. Huang, X. Y. Zhu, and L. Zhao, Attention-based LSTM for aspect-level sentiment classification, in Proc. 2016 Conf. Empirical Methods in Natural Language Proc., Austin, TX, USA, 2016, pp. 606-615.
[13]
D. Y. Tang, B. Qin, and T. Liu, Aspect level sentiment classification with deep memory network, in Proc. 2016 Conf. Empirical Methods in Natural Language Proc., Austin, TX, USA, 2016, pp. 214-224.
[14]
F. F. Fan, Y. S. Feng, and D. Y. Zhao, Multi-grained attention network for aspect-Level sentiment classification, in Proc. 2018 Conf. Empirical Methods in Natural Language Proc., Brussels, Belgium, 2018, pp. 3433-3442.
[15]
P. Chen, Z. Q. Sun, L. D. Bing, and W. Yang, Recurrent attention network on memory for aspect sentiment analysis, in Proc. 2017 Conf. Empirical Methods in Natural Language Proc., Copenhagen, Denmark, 2017, pp. 452-461.
[16]
D. H. Ma, S. J. Li, X. D. Zhang, and H. F. Wang, Interactive attention networks for aspect-level sentiment classification, in Proc. 26th Int. Joint Conf. Artificial Intelligence, Melbourne, Australia, 2017, pp. 4068-4074.
[17]
B. X. Huang, Y. L. Ou, and K. M. Carley, Aspect level sentiment classification with attention-over-attention neural networks, in Int. Conf. Social Computing, Behavioral-Cultural Modeling & Prediction and Behavior Representation in Modeling and Simulation (SBP-BRiMS), R. Thomson, C. Dancy, A. Hyder, and H. Bisgin, Eds. Cham, Germany: Springer, 2018, pp. 197-206.
[18]
L. S. Li, Y. Liu, and A. Q. Zhou, Hierarchical attention based position-aware network for aspect-level sentiment analysis, in Proc. 22nd Conf. Computational Natural Language Learning, Brussels, Belgium, 2018, pp. 181-189.
[19]
Y. W. Song, J. H. Wang, T. Jiang, Z. Y. Liu, and Y. H. Rao, Attentional encoder network for targeted sentiment classification, arXiv preprint arXiv: 1902.09314v2, 2019.
[20]
X. Li, L. D. Bing, W. Lam, and B. Shi, Transformation networks for target-oriented sentiment classification, in Proc. 56th Ann. Meeting of the Association for Computational Linguistics, Melbourne, Australia, 2018, pp. 946-956.
[21]
F. X. Wang, M. Lan, and W. T. Wang, Towards a one-stop solution to both aspect extraction and sentiment analysis tasks with neural multi-task learning, in Proc. 2018 Int. Joint Conf. Neural Networks (IJCNN), Rio de Janeiro, Brazil, 2018, pp. 1-8.
[22]
X. Li, L. D. Bing, P. J. Li, and W. Lam, A unified model for opinion target extraction and target sentiment prediction, Proc. AAAI Conf. Artif. Intell., vol. 33, no. 1, pp. 6714-6721, 2019.
[23]
R. D. He, W. S. Lee, H. T. Ng, and D. Dahlmeier, An interactive multi-task learning network for end-to-end aspect-based sentiment analysis, in Proc. 57th Ann. Meeting of the Association for Computational Linguistics, Florence, Italy, 2019, pp. 504-515.
[24]
H. S. Luo, T. R. Li, B. Liu, and J. B. Zhang, DOER: Dual cross-shared RNN for aspect term-polarity co-extraction, in Proc. 57th Ann. Meeting of the Association for Computational Linguistics, Florence, Italy, 2019, pp. 591-601.
[25]
Y. Kim, Convolutional neural networks for sentence classification, in Proc. 2014 Conf. Empirical Methods in Natural Language Proc., Doha, Qatar, 2014, pp. 1746-1751.
[26]
K. Cho, B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, Learning phrase representations using RNN encoder-decoder for statistical machine translation, in Proc. 2014 Conf. Empirical Methods in Natural Language Processing, Doha, Qatar, 2014, pp. 1724-1734.
[27]
L. Sha, X. D. Zhang, F. Qian, B. B. Chang, and Z. F. Sui, A Multi-view fusion neural network for answer selection, in 32nd AAAI Conf. Artificial Intelligence, New Orleans, LA, USA, 2018, pp. 5422-5429.
[28]
G. Andrew, R. Arora, J. Bilmes, and K. Livescu, Deep canonical correlation analysis, in Proc. 30th Int. Conf. Machine Learning, Atlanta, GA, USA, 2013, pp. 1247-1255.
[29]
S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural Comput., vol. 9, no. 8, pp. 1735-1780, 1997.
[30]
J. Weston, S. Chopra, and A. Bordes, Memory networks, arXiv preprint arXiv: 1410.3916, 2014.
[31]
Y. M. Cui, Z. P. Chen, S. Wei, S. J. Wang, T. Liu, and G. P. Hu, Attention-over-attention neural networks for reading comprehension, in Proc. 55th Ann. Meeting of the Association for Computational Linguistics, Vancouver, Canada, 2017, pp. 593-602.
[32]
J. Pennington, R. Socher, and C. D. Manning, Glove: Global vectors for word representation, in Proc. 2014 Conf. Empirical Methods in Natural Language Proc., Doha, Qatar, 2014, pp. 1532-1543.
[33]
K. M. He, X. Y. Zhang, S. Q. Ren, and J. Sun, Deep residual learning for image recognition, in Proc. 2016 IEEE Conf. Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016, pp. 770-778.
[34]
M. Pontiki, D. Galanis, J. Pavlopoulos, H. Papageorgiou, I. Androutsopoulos, and S. Manandhar, Semeval-2014 task 4: Aspect based sentiment analysis, in Proc. 8th Int. Workshop on Semantic Evaluation (SemEval 2014), Dublin, Ireland, 2014, pp. 27-35.
[35]
M. Mitchell, J. Aguilar, T. Wilson, and B. Van Durme, Open domain targeted sentiment, in Proc. 2013 Conf. Empirical Methods in Natural Language Proc., Seattle, WA, USA, 2013, pp. 1643-1654.
[36]
M. S. Zhang, Y. Zhang, and D. T. Vo, Neural networks for open domain targeted sentiment, in Proc. 2015 Conf. Empirical Methods in Natural Language Proc., Lisbon, Portugal, 2015, pp. 612-621.
[37]
H. Li and W. Lu, Learning latent sentiment scopes for entity-level sentiment analysis, in Proc. 31st AAAI Conf. Artificial Intelligence, San Francisco, CA, USA, 2017, pp. 3482-3489.
[38]
G. Lample, M. Ballesteros, S. Subramanian, K. Kawakami, and C. Dyer, Neural architectures for named entity recognition, in Proc. 2016 Conf. North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, CA, USA, 2016, pp. 260-270.
[39]
X. Z. Ma and E. Hovy, End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF, in Proc. 54th Annu. Meeting of the Association for Computational Linguistics, Berlin, Germany, 2016, pp. 1064-1074.
[40]
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł Kaiser, and I. Polosukhin, Attention is all you need, in Proc. 31st Conf. Neural Information Proc. Systems, Long Beach, CA, USA, 2017, pp. 5998-6008.
[41]
X. Glorot and Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, in Proc. 13th Int. Conf. Artificial Intelligence and Statistics, Sardinia, Italy, vol. 9, pp. 249-256.
[42]
K. M. He, X. Y. Zhang, S. Q. Ren, and J. Sun, Delving deep into rectifiers: surpassing human-level performance on ImageNet classification, in 2015 IEEE Int. Conf. Computer Vision (ICCV), Santiago, Chile, 2015, pp. 1026-1034.
[43]
F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, The graph neural network model, IEEE Trans. Neural Networks, vol. 20, no. 1, pp. 61-80, 2009.
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 03 January 2021
Accepted: 25 January 2021
Published: 12 May 2021
Issue date: September 2021

Copyright

© The author(s) 2021

Acknowledgements

This work was supported by the National Natural Science Foundation of China (No. 61976247).

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Reprints and Permission requests may be sought directly from editorial office.

Return