Journal Home > Volume 2 , Issue 3

Sentiment analysis refers to the automatic collection, aggregation, and classification of data collected online into different emotion classes. While most of the work related to sentiment analysis of texts focuses on the binary and ternary classification of these data, the task of multi-class classification has received less attention. Multi-class classification has always been a challenging task given the complexity of natural languages and the difficulty of understanding and mathematically "quantifying" how humans express their feelings. In this paper, we study the task of multi-class classification of online posts of Twitter users, and show how far it is possible to go with the classification, and the limitations and difficulties of this task. The proposed approach of multi-class classification achieves an accuracy of 60.2% for 7 different sentiment classes which, compared to an accuracy of 81.3% for binary classification, emphasizes the effect of having multiple classes on the classification performance. Nonetheless, we propose a novel model to represent the different sentiments and show how this model helps to understand how sentiments are related. The model is then used to analyze the challenges that multi-class classification presents and to highlight possible future enhancements to multi-class classification accuracy.


menu
Abstract
Full text
Outline
About this article

Multi-Class Sentiment Analysis on Twitter: Classification Performance and Challenges

Show Author's information Mondher Bouazizi( )Tomoaki Ohtsuki
Department of Information and Computer Science, Keio University, Yokohama 223-8542, Japan.

Abstract

Sentiment analysis refers to the automatic collection, aggregation, and classification of data collected online into different emotion classes. While most of the work related to sentiment analysis of texts focuses on the binary and ternary classification of these data, the task of multi-class classification has received less attention. Multi-class classification has always been a challenging task given the complexity of natural languages and the difficulty of understanding and mathematically "quantifying" how humans express their feelings. In this paper, we study the task of multi-class classification of online posts of Twitter users, and show how far it is possible to go with the classification, and the limitations and difficulties of this task. The proposed approach of multi-class classification achieves an accuracy of 60.2% for 7 different sentiment classes which, compared to an accuracy of 81.3% for binary classification, emphasizes the effect of having multiple classes on the classification performance. Nonetheless, we propose a novel model to represent the different sentiments and show how this model helps to understand how sentiments are related. The model is then used to analyze the challenges that multi-class classification presents and to highlight possible future enhancements to multi-class classification accuracy.

Keywords: machine learning, sentiment analysis, Twitter

References(18)

[1]
M. A. Cabanlit and K. J. Espinosa, Optimizing N-gram based text feature selection in sentiment analysis for commercial products in Twitter through polarity lexicons, in Proc. 5th Int. Conf. Information, Intelligence, Systems and Applications, Chania, Greece, 2014, pp. 94-97.
DOI
[2]
U. R. Hodeghatta, Sentiment analysis of Hollywood movies on Twitter, in Proc. 2013 IEEE/ACM Int. Conf. on Advances in Social Networks Analysis and Mining, Niagara Falls, Canada, 2013, pp. 1401-1404.
DOI
[3]
J. M. Soler, F. Cuartero, and M. Roblizo, Twitter as a tool for predicting elections results, in Proc. 2012 IEEE/ACM Int. Conf. on Advances in Social Networks Analysis and Mining, Istanbul, Turkey, 2012, pp. 1194-1200.
DOI
[4]
K. Ghag and K. Shah, Comparative analysis of the techniques for sentiment analysis, in Proc. 2013 Int. Conf. on Advances in Technology and Engineering, Mumbai, India, 2013, pp. 1-7.
DOI
[5]
K. H. Y. Lin, C. H. Yang, and H. H. Chen, What emotions do news articles trigger in their readers? in Proc. 30th Annu. Int. ACM SIGIR Conf. on Research and Development in Information Retrieval, Amsterdam, Netherlands, 2007, pp. 733-734.
DOI
[6]
K. H. Y. Lin, C. H. Yang, and H. H. Chen, Emotion classification of online news articles from the reader’s perspective, in Proc. 2008 IEEE/WIC/ACM Int. Conf. on Web Intelligence and Intelligent Agent Technology, Sydney, Australia, 2008, pp. 220-226.
DOI
[7]
L. Ye, R. F. Xu, and J. Xu, Emotion prediction of news articles from reader’s perspective based on multi-label classification, in Proc. 2012 Int. Conf. on Machine Learning and Cybernetics, Xi’an, China, 2012, pp. 2019-2024.
[8]
W. B. Liang, H. C. Wang, Y. A. Chu, and C. H. Wu, Emoticon recommendation in microblog using affective trajectory model, in Proc. 2014 Asia-Pacific Signal and Information Processing Association Annu. Summit and Conf., Chiang Mai, Thailand, 2014, pp. 1-5.
DOI
[9]
B. Sriram, D. Fuhry, E. Demir, H. Ferhatosmanoglu, and M. Demirbas, Short text classification in twitter to improve information filtering, in Proc. 33rd Int. ACM SIGIR Conf. on Research and Development in Information Retrieval, Geneva, Switzerland, 2010, pp. 841-842.
DOI
[10]
M. Boia, B. Faltings, C. C. Musat, and P. Pu, A:) is worth a thousand words: How people attach sentiment to emoticons and words in tweets, in Proc. 2013 Int. Conf. on Social Computing, Alexandria, VA, USA, 2013, pp. 345-350.
DOI
[11]
K. Manuel, K. V. Indukuri, and P. R. Krishna, Analyzing internet slang for sentiment mining, in Proc. 2010 2nd Vaagdevi Int. Conf. on Information Technology for Real World Problems, Warangal, India, 2010, pp. 9-11.
DOI
[12]
Y. H. P. P. Priyadarshana, K. I. H. Gunathunga, K. K. A. Nipuni, N. Perera, L. Ranathunga, P. M. Karunaratne, and T. M. Thanthriwatta, Sentiment analysis: Measuring sentiment strength of call centre conversations, in Proc. 2015 IEEE Int. Conf. on Electrical, Computer and Communication Technologies, Coimbatore, India, 2015, pp. 1-9.
DOI
[13]
R. Srivastava and M. P. S. Bhatia, Quantifying modified opinion strength: A fuzzy inference system for Sentiment Analysis, in Proc. 2012 Int. Conf. on Advances in Computing, Communications and Informatics, Mysore, India, 2013, pp. 1512-1519.
DOI
[14]
M. Bouazizi and T. Ohtsuki, Sentiment analysis: From binary to multi-class classification: A pattern-based approach for multi-class sentiment analysis in Twitter, in Proc. 2016 IEEE Int. Conf. on Communications, Kuala Lumpur, Malaysia, 2016, pp. 1-6.
DOI
[15]
M. Bouazizi and T. Ohtsuki, A pattern-based approach for multi-class sentiment analysis in twitter, IEEE Access, vol. 5, pp. 20617-20639, 2017.
[16]
M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten, The WEKA data mining software: An update, ACM SIGKDD Explor. Newsl., vol. 11, no. 1, pp. 10-18, 2009.
[17]
L. Breiman, Random forests, Mach. Learn., vol. 45, no. 1, pp. 5-32, 2001.
[18]
D. Davidov, O. Tsur, and A. Rappoport, Semi-supervised recognition of sarcastic sentences in Twitter and Amazon, in Proc. 14th Conf. on Computational Natural Language Learning, Uppsala, Sweden, 2010, pp. 107-116.
Publication history
Copyright
Rights and permissions

Publication history

Received: 10 November 2018
Accepted: 13 February 2019
Published: 04 April 2019
Issue date: September 2019

Copyright

© The author(s) 2019

Rights and permissions

Return