EKBSA: A Chinese Sentiment Analysis Model by Enhancing K-BERT

Pre-trained language models (PLMs), such as BERT, have achieved good results on many natural language processing (NLP) tasks. Recently, some studies have attempted to integrate factual knowledge into PLMs to adapt to various downstream tasks. For sentiment analysis tasks, sentiment knowledge, such as sentiment words, plays a significant role in determining the sentiment tendencies of texts. For Chinese sentiment analysis, historical stories and fables imbue words with richer connotations and more complex sentiments than those typically found in English, which makes sentiment knowledge injection necessary. But clearly, this knowledge has not been fully considered. In this paper, we propose EKBSA, a Chinese sentiment analysis model, which is based on the K-BERT model and utilizes a sentiment knowledge graph to achieve better results on sentiment analysis tasks. To construct a high-quality sentiment knowledge graph, we collect a large number of sentiment words by combining several existing sentiment lexica. Moreover, in order to understand texts better, we enhance local attention through syntactic analysis and direct to EKBSA focus more on syntactically relevant words. EKBSA is compatible with BERT and existing structural knowledge. Experimental results show that EKBSA achieves better performance on Chinese sentiment analysis tasks. Built upon EKBSA, we further change the general attention to the context attention and propose Context EKBSA, so that the model can adapt to sentiment analysis tasks in Chinese conversations and achieve good performance.