AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Journal of Social Computing Article
PDF (1.6 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Prediction of Academic Performance of Students in Online Live Classroom Interactions —An Analysis Using Natural Language Processing and Deep Learning Methods

Yuanyi Zhen1, Jar-Der Luo1( ), Hui Chen2
Laboratory of Computational Social Science and National Governance, Tsinghua University, Beijing 100084, China
School of Chinese Language and Literature, Beijing Foreign Studies University, Beijing 100089, China
Show Author Information

Abstract

Prior studies have shown the importance of classroom dialogue in academic performance, through which knowledge construction and social interaction among students take place. However, most of them were based on small scale or qualitative data, and few has explored the availability and potential of big data collected from online classrooms. To address this issue, this paper analyzes dialogues in live classrooms of a large online learning platform in China based on natural language processing techniques. The features of interactive types and emotional expression are extracted from classroom dialogues. We then develop neural network models based on these features to predict high- and low-academic performing students, and employ interpretable AI (artificial intelligence) techniques to determine the most important predictors in the prediction models. In both STEM (science, technology, engineering, mathematics) and non-STEM courses, it is found that high-performing students consistently exhibit more positive emotion, cognition and off-topic dialogues in all stages of the lesson than low-performing students. However, while the metacognitive dialogue illustrates its importance in non-STEM courses, this effect cannot be found in STEM courses. While high-performing students in non-STEM courses show negative emotion in the last stage of lessons, STEM students show positive emotion.

Keywords

natural language processingdeep learningacademic performance predictionlive classroom dialogueemotional expressioninteractive type

References

[1]

J. E. Knowles, Of needles and haystacks: Building an accurate statewide dropout early warning system in wisconsin, J. Educ. Data Min., vol. 7, no. 3, pp. 18–67, 2015.
Google Scholar
[2]

S. Lee and J. Y. Chung, The machine learning-based dropout early warning system for improving the performance of dropout prediction, Appl. Sci., vol. 9, no. 15, p. 3093, 2019.
Crossref Google Scholar
[3]

L. Yan, A. Whitelock-Wainwright, Q. Guan, G. Wen, D. Gašević, and G. Chen, Students’ experience of online learning during the COVID-19 pandemic: A province-wide survey study, Br. J. Educ. Technol., vol. 52, no. 5, pp. 2038–2057, 2021.
Crossref Google Scholar
[4]
H. Li, W. Ding, and Z. Liu, Identifying at-risk K-12 students in multimodal online environments: A machine learning approach, arXiv preprint arXiv: 2003.09670, 2020.
[5]

S. Hennessy, S. Rojas-Drummond, R. Higham, A. M. Márquez, F. Maine, R. M. Ríos, R. García-Carrión, O. Torreblanca, and M. J. Barrera, Developing a coding scheme for analysing classroom dialogue across educational contexts, Learn. Cult. Soc. Interact., vol. 9, pp. 16–44, 2016.
Crossref Google Scholar
[6]

M. Zembylas, Adult learners’ emotions in online learning, Distance Educ., vol. 29, no. 1, pp. 71–87, 2008.
Crossref Google Scholar
[7]
Z. Liu, and Y. Zhang, A semantic role mining and learning performance prediction method in MOOCs, presented at Asia-Pacific Web (APWeb) and Web-Age Information Management (WAIM) Joint Int. Conf. Web and Big Data, Macau, China, 2018, pp. 259–269.
Crossref
[8]

W. Zou, X. Hu, Z. Pan, C. Li, Y. Cai, and M. Liu, Exploring the relationship between social presence and learners’ prestige in MOOC discussion forums using automated content analysis and social network analysis, Comput. Hum. Behav., vol. 115, p. 106582, 2021.
Crossref Google Scholar
[9]

C. Howe and M. Abedin, Classroom dialogue: A systematic review across four decades of research, Camb. J. Educ., vol. 43, no. 3, pp. 325–356, 2013.
Crossref Google Scholar
[10]
R. J. Alexander, Essays on Pedagogy. New York, NY, USA: Routledge, 2008.
[11]
J. G. Greeno, Classroom talk sequences and learning, in Socializing Intelligence Through Academic Talk and Dialogue, L. B. Resnick, C. S. Asterhan, and S. N. Clarke, eds. New York, NY, USA: American Educational Research Association, 2015, pp. 255–262.
Crossref
[12]

H. Muhonen, E. Pakarinen, A. M. Poikkeus, M. K. Lerkkanen, and H. Rasku-Puttonen, Quality of educational dialogue and association with students’ academic performance, Learn. Instr., vol. 55, pp. 67–79, 2018.
Crossref Google Scholar
[13]
M. Baker, S. Järvelä, J. Andriessen, Affective Learning Together: Social and Emotional Dimensions of Collaborative Learning. New York, NY, USA: Routledge, 2013.
Crossref
[14]

G. Zhu, W. Xing, S. Costa, M. Scardamalia, and B. Pei, Exploring emotional and cognitive dynamics of Knowledge Building in grades 1 and 2, User Model. User Adapt. Interact., vol. 29, no. 4, pp. 789–820, 2019.
Crossref Google Scholar
[15]
P. R. Kleinginna Jr and A. M. Kleinginna, A categorized list of emotion definitions, with suggestions for a consensual definition, Motiv. Emot., vol. 5, no. 4, pp. 345–379, 1981.
Crossref
[16]

K. R. Scherer, What are emotions? And how can they be measured, Soc. Sci. Inf., vol. 44, no. 4, pp. 695–729, 2005.
Crossref Google Scholar
[17]

A. Bakhtiar, E. A. Webster, and A. F. Hadwin, Regulation and socio-emotional interactions in a positive and a negative group climate, Metacognition Learn., vol. 13, no. 1, pp. 57–90, 2018.
Crossref Google Scholar
[18]

M. Cleveland-Innes and P. Campbell, Emotional presence, learning, and the online learning environment, Int. Rev. Res. Open Distrib. Learn., vol. 13, no. 4, p. 269, 2012.
Crossref Google Scholar
[19]

A. R. Damasio, Descartes’ error and the future of human life, Sci. Am., vol. 271, no. 4, p. 144, 1994.
Crossref Google Scholar
[20]
R. Pekrun and L. Linnenbink-Garcia, Academic emotions and student engagement, in Handbook of Research on Student Engagement, S. L. Christenson, A. L. Reschly, and C. Wylie, eds. New York, NY, USA: Springer, 2012, pp. 259–282.
Crossref
[21]

L. Yorks and E. Kasl, Toward a theory and practice for whole-person learning: Reconceptualizing experience and the role of affect, Adult Educ. Q., vol. 52, no. 3, pp. 176–192, 2002.
Crossref Google Scholar
[22]
R. Pekrun, A social cognitive, control–value theory of achievement emotions, in Motivational Psychology of Human Development, J. Heckhausen, ed. Oxford, UK: Elsevier, pp. 143–163, 2000.
Crossref
[23]

R. Pekrun, S. Lichtenfeld, H. W. Marsh, K. Murayama, and T. Goetz, Achievement emotions and academic performance: Longitudinal models of reciprocal effects, Child Dev., vol. 88, no. 5, pp. 1653–1670, 2017.
Crossref Google Scholar
[24]

R. S. J. D. Baker, S. K. D'Mello, M. M. T. Rodrigo, and A. C. Graesser, Better to be frustrated than bored: The incidence, persistence, and impact of learners’ cognitive-affective states during interactions with three different computer-based learning environments, Int. J. Hum. Comput. Stud., vol. 68, no. 4, pp. 223–241, 2010.
Crossref Google Scholar
[25]
Z. A. Pardos, R. S. Baker, M. O. San Pedro, S. M. Gowda, and S. M. Gowda, Affective states and state tests: Investigating how affect throughout the school year predicts end of year learning outcomes, in Proc. 3rd Int. Conf. learning analytics and knowledge, Leuven, Belgium, 2013, pp. 117–124.
Crossref
[26]

S. D’Mello, B. Lehman, R. Pekrun, and A. Graesser, Confusion can be beneficial for learning, Learn. Instr., vol. 29, pp. 153–170, 2014.
Crossref Google Scholar
[27]
M. Worsley, P. Blikstein, Using learning analytics to study cognitive disequilibrium in a complex learning environment, in Proc. 5th Int. Conf. Learning Analytics and Knowledge, Poughkeepsie, NY, USA, 2015, pp. 426–427.
Crossref
[28]

L. Zheng, J. Niu, and L. Zhong, Effects of a learning analytics-based real-time feedback approach on knowledge elaboration, knowledge convergence, interactive relationships and group performance in CSCL, Br. J. Educ. Technol., vol. 53, no. 1, pp. 130–149, 2022.
Crossref Google Scholar
[29]

L. Przybilla, K. Klinker, M. Kauschinger, and H. Krcmar, Stray off topic to stay on topic: Preserving interaction and team morale in a highly collaborative course while at a distance, Commun. Assoc. Inf. Syst., vol. 48, no. 1, pp. 177–184, 2021.
Crossref Google Scholar
[30]

J. H. Flavell, Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry, Am. Psychol., vol. 34, no. 10, pp. 906–911, 1979.
Crossref Google Scholar
[31]

M. E. Martinez, What is metacognition, Phi Delta Kappan, vol. 87, no. 9, pp. 696–699, 2006.
Crossref Google Scholar
[32]

C. D. Zepeda, C. O. Hlutkowsky, A. C. Partika, and T. J. Nokes-Malach, Identifying teachers’ supports of metacognition through classroom talk and its relation to growth in conceptual learning, J. Educ. Psychol., vol. 111, no. 3, pp. 522–541, 2019.
Crossref Google Scholar
[33]

P. H. Winne and J. C. Nesbit, The psychology of academic achievement, Annu. Rev. Psychol., vol. 61, pp. 653–678, 2010.
Crossref Google Scholar
[34]

A. Cadamuro, E. Bisagno, G. A. Di Bernardo, L. Vezzali, and A. Versari, Making the school Smart: the interactive whiteboard against disparities in children stemming from low metacognitive skills, J. E-Learn. Knowl. Soc., vol. 16, no. 1, pp. 33–43, 2020.
Google Scholar
[35]

C. M. Roebers, S. S. Krebs, and T. Roderer, Metacognitive monitoring and control in elementary school children: Their interrelations and their role for test performance, Learn. Individ. Differ., vol. 29, pp. 141–149, 2014.
Crossref Google Scholar
[36]

H. L. Swanson, Influence of metacognitive knowledge and aptitude on problem solving, J. Educ. Psychol., vol. 82, no. 2, pp. 306–314, 1990.
Crossref Google Scholar
[37]

M. V. J. Veenman, R. Kok, and A. W. Blöte, The relation between intellectual and metacognitive skills in early adolescence, Instr. Sci., vol. 33, no. 3, pp. 193–211, 2005.
Crossref Google Scholar
[38]

Z. R. Mevarech and C. Amrany, Immediate and delayed effects of meta-cognitive instruction on regulation of cognition and mathematics achievement, Metacognition Learn., vol. 3, no. 2, pp. 147–157, 2008.
Crossref Google Scholar
[39]
R. J. Yinger, A study of teacher planning, Elem. Sch. J. , vol. 80, no. 3, pp. 107–127, 1980.
Crossref
[40]
M. Hunter, Knowing, teaching and supervising, in Using What We Know About Reading, P. Hosford, ed. Alexandria, Egypt: Association for Supervision and Curriculum Development, 1984, pp. 169–203.
[41]

A. P. Johnson, It’s time for madeline hunter to go: A new look at lesson plan design, Action Teach. Educ., vol. 22, no. 1, pp. 72–78, 2000.
Crossref Google Scholar
[42]
T. S. Farrell, Lesson planning, in Methodology in Language Teaching: An Anthology of Current Practice, J. C. Richards and W. A. Renandya, Eds. New York, NY, USA: Cambridge University Press, 2002, pp. 30–39.
Crossref
[43]

G. Tonguç and B. O. Ozkara, Automatic recognition of student emotions from facial expressions during a lecture, Comput. Educ., vol. 148, p. 103797, 2020.
Crossref Google Scholar
[44]
M. Ford, C. T. Baer, D. Xu, U. Yapanel, and S. Gray, The LENATM language environment analysis system: Audio specifications of the DLP-0121, Technical report, LENA Foundation, Boulder, CO, USA, 2008.
[45]

Z. Wang, K. Miller, and K. Cortina, Using the LENA in teacher training: Promoting student involvement through automated feedback, Unterrichtswissenschaft, vol. 4, pp. 290–305, 2013.
Google Scholar
[46]

Z. Wang, X. Pan, K. F. Miller, and K. S. Cortina, Automatic classification of activities in classroom discourse, Comput. Educ., vol. 78, pp. 115–123, 2014.
Crossref Google Scholar
[47]
D. Jiang, Y. Chen, and A. Garg, A hybrid method for overlapping speech detection in classroom environment, Comput. Appl. Eng. Educ., vol. 26, no. 1, pp. 171–180, 2018.
Crossref
[48]

Y. Song, S. Lei, T. Hao, Z. Lan, and Y. Ding, Automatic classification of semantic content of classroom dialogue, J. Educ. Comput. Res., vol. 59, no. 3, pp. 496–521, 2021.
Crossref Google Scholar
[49]
P. M. Moreno-Marcos, P. J. Muñoz-Merino, C. Alario-Hoyos, I. Estévez-Ayres, and C. Delgado Kloos, Analysing the predictive power for anticipating assignment grades in a massive open online course, Behav. Inf. Technol., vol. 37, nos. 10–11, pp. 1021–1036, 2018.
Crossref
[50]
K. Mrhar, L. Benhiba, S. Bourekkache, and M. Abik, A Bayesian CNN-LSTM model for sentiment analysis in massive open online courses MOOCs, Int. J. Emerg. Technol. Learn. , vol. 16, no. 23, pp. 216–232, 2021.
Crossref
[51]

S. Wachter, B. Mittelstadt, C. Russell, Counterfactual explanations without opening the black box: Automated decisions and the GDPR, SSRN Electron. J., vol. 31, no. 2, pp. 841–887, 2017.
Crossref Google Scholar
[52]
A. Chatzimparmpas, R. M. Martins, I. Jusufi, and A. Kerren, A survey of surveys on the use of visualization for interpreting machine learning models, Inf. Vis. , vol. 19, no. 3, pp. 207–233, 2020.
Crossref
[53]

G. Plumb, D. Molitor, A. S. Talwalkar, Model Agnostic Supervised Local Explanations, NIPS, vol. 31, pp. 2520–2529, 2018.
Google Scholar
[54]
M. T. Ribeiro, S. Singh, C. Guestrin, Why should I trust you?: Explaining the predictions of any classifier, in Proc. 22nd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, San Francisco, CA, USA, 2016, pp. 1135–1144.
Crossref
[55]
S. M. Lundberg, S. I. Lee, A unified approach to interpreting model predictions, NIPS, vol. 30, pp. 4768–4777, 2017.
[56]

S. Karlos, G. Kostopoulos, and S. Kotsiantis, Predicting and interpreting students’ grades in distance higher education through a semi-regression method, Appl. Sci., vol. 10, no. 23, p. 8413, 2020.
Crossref Google Scholar
[57]
S. Kim, W. Kim, Y. Jang, S. Choi, H. Jung, and H. Kim, Student knowledge prediction for teacher-student interaction, in Proc. AAAI Conf. Artif. Intell. , vol. 35, no. 17, pp. 15560–15568, 2021.
Crossref
[58]
J. Devlin, M. Chang, K. Lee, and K. Toutanova, BERT: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv: 1810.04805, 2018.
[59]

J. Hoth, G. Kaiser, A. Busse, M. Doehrmann, J. Koenig, and S. Blömeke, Professional competences of teachers for fostering creativity and supporting high-achieving students, ZDM Math. Educ., vol. 49, no. 1, pp. 107–120, 2017.
Crossref Google Scholar
[60]
J. T. Guthrie and A. Wigfield, Engagement and motivation in reading, in Handbook of Reading Research, P. D. Barr, ed. New York, NY, USA: Lawrence Erlbaum Associates, 2000, pp. 403–422.
[61]

X. Xu, J. Wang, H. Peng, and R. Wu, Prediction of academic performance associated with Internet usage behaviors using machine learning algorithms, Comput. Hum. Behav., vol. 98, pp. 166–173, 2019.
Crossref Google Scholar
[62]

C. F. Rodríguez-Hernández, M. Musso, E. Kyndt, and E. Cascallar, Artificial neural networks in academic performance prediction: Systematic implementation and predictor evaluation, Comput. Educ., vol. 2, p. 100018, 2021.
Crossref Google Scholar
[63]

L. Zheng, M. Long, J. Niu, and L. Zhong, An automated group learning engagement analysis and feedback approach to promoting collaborative knowledge building, group performance, and socially shared regulation in CSCL, Int. J. Comput. Support. Collab. Learn., vol. 18, no. 1, pp. 101–133, 2023.
Crossref Google Scholar
Journal of Social Computing
Volume 4 Issue 1,
March 2023
Pages 12-29
DOI: 10.23919/JSC.2023.0007
Cite this article:
Zhen Y, Luo J-D, Chen H. Prediction of Academic Performance of Students in Online Live Classroom Interactions —An Analysis Using Natural Language Processing and Deep Learning Methods. Journal of Social Computing, 2023, 4(1): 12-29. https://doi.org/10.23919/JSC.2023.0007

4197

Views

598

Downloads

5

Crossref

8

Scopus

Altmetrics
Received: 13 February 2023
Revised: 01 June 2023
Accepted: 06 June 2023
Published: 30 March 2023
© The author(s) 2023.

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
Return