AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
PDF (1.6 MB)
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Prediction of Academic Performance of Students in Online Live Classroom Interactions —An Analysis Using Natural Language Processing and Deep Learning Methods

Yuanyi Zhen1Jar-Der Luo1( )Hui Chen2
Laboratory of Computational Social Science and National Governance, Tsinghua University, Beijing 100084, China
School of Chinese Language and Literature, Beijing Foreign Studies University, Beijing 100089, China
Show Author Information

Abstract

Prior studies have shown the importance of classroom dialogue in academic performance, through which knowledge construction and social interaction among students take place. However, most of them were based on small scale or qualitative data, and few has explored the availability and potential of big data collected from online classrooms. To address this issue, this paper analyzes dialogues in live classrooms of a large online learning platform in China based on natural language processing techniques. The features of interactive types and emotional expression are extracted from classroom dialogues. We then develop neural network models based on these features to predict high- and low-academic performing students, and employ interpretable AI (artificial intelligence) techniques to determine the most important predictors in the prediction models. In both STEM (science, technology, engineering, mathematics) and non-STEM courses, it is found that high-performing students consistently exhibit more positive emotion, cognition and off-topic dialogues in all stages of the lesson than low-performing students. However, while the metacognitive dialogue illustrates its importance in non-STEM courses, this effect cannot be found in STEM courses. While high-performing students in non-STEM courses show negative emotion in the last stage of lessons, STEM students show positive emotion.

References

[1]

J. E. Knowles, Of needles and haystacks: Building an accurate statewide dropout early warning system in wisconsin, J. Educ. Data Min., vol. 7, no. 3, pp. 18–67, 2015.

[2]

S. Lee and J. Y. Chung, The machine learning-based dropout early warning system for improving the performance of dropout prediction, Appl. Sci., vol. 9, no. 15, p. 3093, 2019.

[3]

L. Yan, A. Whitelock-Wainwright, Q. Guan, G. Wen, D. Gašević, and G. Chen, Students’ experience of online learning during the COVID-19 pandemic: A province-wide survey study, Br. J. Educ. Technol., vol. 52, no. 5, pp. 2038–2057, 2021.

[4]
H. Li, W. Ding, and Z. Liu, Identifying at-risk K-12 students in multimodal online environments: A machine learning approach, arXiv preprint arXiv: 2003.09670, 2020.
[5]

S. Hennessy, S. Rojas-Drummond, R. Higham, A. M. Márquez, F. Maine, R. M. Ríos, R. García-Carrión, O. Torreblanca, and M. J. Barrera, Developing a coding scheme for analysing classroom dialogue across educational contexts, Learn. Cult. Soc. Interact., vol. 9, pp. 16–44, 2016.

[6]

M. Zembylas, Adult learners’ emotions in online learning, Distance Educ., vol. 29, no. 1, pp. 71–87, 2008.

[7]
Z. Liu, and Y. Zhang, A semantic role mining and learning performance prediction method in MOOCs, presented at Asia-Pacific Web (APWeb) and Web-Age Information Management (WAIM) Joint Int. Conf. Web and Big Data, Macau, China, 2018, pp. 259–269.
[8]

W. Zou, X. Hu, Z. Pan, C. Li, Y. Cai, and M. Liu, Exploring the relationship between social presence and learners’ prestige in MOOC discussion forums using automated content analysis and social network analysis, Comput. Hum. Behav., vol. 115, p. 106582, 2021.

[9]

C. Howe and M. Abedin, Classroom dialogue: A systematic review across four decades of research, Camb. J. Educ., vol. 43, no. 3, pp. 325–356, 2013.

[10]
R. J. Alexander, Essays on Pedagogy. New York, NY, USA: Routledge, 2008.
[11]
J. G. Greeno, Classroom talk sequences and learning, in Socializing Intelligence Through Academic Talk and Dialogue, L. B. Resnick, C. S. Asterhan, and S. N. Clarke, eds. New York, NY, USA: American Educational Research Association, 2015, pp. 255–262.
[12]

H. Muhonen, E. Pakarinen, A. M. Poikkeus, M. K. Lerkkanen, and H. Rasku-Puttonen, Quality of educational dialogue and association with students’ academic performance, Learn. Instr., vol. 55, pp. 67–79, 2018.

[13]
M. Baker, S. Järvelä, J. Andriessen, Affective Learning Together: Social and Emotional Dimensions of Collaborative Learning. New York, NY, USA: Routledge, 2013.
[14]

G. Zhu, W. Xing, S. Costa, M. Scardamalia, and B. Pei, Exploring emotional and cognitive dynamics of Knowledge Building in grades 1 and 2, User Model. User Adapt. Interact., vol. 29, no. 4, pp. 789–820, 2019.

[15]
P. R. Kleinginna Jr and A. M. Kleinginna, A categorized list of emotion definitions, with suggestions for a consensual definition, Motiv. Emot., vol. 5, no. 4, pp. 345–379, 1981.
[16]

K. R. Scherer, What are emotions? And how can they be measured, Soc. Sci. Inf., vol. 44, no. 4, pp. 695–729, 2005.

[17]

A. Bakhtiar, E. A. Webster, and A. F. Hadwin, Regulation and socio-emotional interactions in a positive and a negative group climate, Metacognition Learn., vol. 13, no. 1, pp. 57–90, 2018.

[18]

M. Cleveland-Innes and P. Campbell, Emotional presence, learning, and the online learning environment, Int. Rev. Res. Open Distrib. Learn., vol. 13, no. 4, p. 269, 2012.

[19]

A. R. Damasio, Descartes’ error and the future of human life, Sci. Am., vol. 271, no. 4, p. 144, 1994.

[20]
R. Pekrun and L. Linnenbink-Garcia, Academic emotions and student engagement, in Handbook of Research on Student Engagement, S. L. Christenson, A. L. Reschly, and C. Wylie, eds. New York, NY, USA: Springer, 2012, pp. 259–282.
[21]

L. Yorks and E. Kasl, Toward a theory and practice for whole-person learning: Reconceptualizing experience and the role of affect, Adult Educ. Q., vol. 52, no. 3, pp. 176–192, 2002.

[22]
R. Pekrun, A social cognitive, control–value theory of achievement emotions, in Motivational Psychology of Human Development, J. Heckhausen, ed. Oxford, UK: Elsevier, pp. 143–163, 2000.
[23]

R. Pekrun, S. Lichtenfeld, H. W. Marsh, K. Murayama, and T. Goetz, Achievement emotions and academic performance: Longitudinal models of reciprocal effects, Child Dev., vol. 88, no. 5, pp. 1653–1670, 2017.

[24]

R. S. J. D. Baker, S. K. D'Mello, M. M. T. Rodrigo, and A. C. Graesser, Better to be frustrated than bored: The incidence, persistence, and impact of learners’ cognitive-affective states during interactions with three different computer-based learning environments, Int. J. Hum. Comput. Stud., vol. 68, no. 4, pp. 223–241, 2010.

[25]
Z. A. Pardos, R. S. Baker, M. O. San Pedro, S. M. Gowda, and S. M. Gowda, Affective states and state tests: Investigating how affect throughout the school year predicts end of year learning outcomes, in Proc. 3rd Int. Conf. learning analytics and knowledge, Leuven, Belgium, 2013, pp. 117–124.
[26]

S. D’Mello, B. Lehman, R. Pekrun, and A. Graesser, Confusion can be beneficial for learning, Learn. Instr., vol. 29, pp. 153–170, 2014.

[27]
M. Worsley, P. Blikstein, Using learning analytics to study cognitive disequilibrium in a complex learning environment, in Proc. 5th Int. Conf. Learning Analytics and Knowledge, Poughkeepsie, NY, USA, 2015, pp. 426–427.
[28]

L. Zheng, J. Niu, and L. Zhong, Effects of a learning analytics-based real-time feedback approach on knowledge elaboration, knowledge convergence, interactive relationships and group performance in CSCL, Br. J. Educ. Technol., vol. 53, no. 1, pp. 130–149, 2022.

[29]

L. Przybilla, K. Klinker, M. Kauschinger, and H. Krcmar, Stray off topic to stay on topic: Preserving interaction and team morale in a highly collaborative course while at a distance, Commun. Assoc. Inf. Syst., vol. 48, no. 1, pp. 177–184, 2021.

[30]

J. H. Flavell, Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry, Am. Psychol., vol. 34, no. 10, pp. 906–911, 1979.

[31]

M. E. Martinez, What is metacognition, Phi Delta Kappan, vol. 87, no. 9, pp. 696–699, 2006.

[32]

C. D. Zepeda, C. O. Hlutkowsky, A. C. Partika, and T. J. Nokes-Malach, Identifying teachers’ supports of metacognition through classroom talk and its relation to growth in conceptual learning, J. Educ. Psychol., vol. 111, no. 3, pp. 522–541, 2019.

[33]

P. H. Winne and J. C. Nesbit, The psychology of academic achievement, Annu. Rev. Psychol., vol. 61, pp. 653–678, 2010.

[34]

A. Cadamuro, E. Bisagno, G. A. Di Bernardo, L. Vezzali, and A. Versari, Making the school Smart: the interactive whiteboard against disparities in children stemming from low metacognitive skills, J. E-Learn. Knowl. Soc., vol. 16, no. 1, pp. 33–43, 2020.

[35]

C. M. Roebers, S. S. Krebs, and T. Roderer, Metacognitive monitoring and control in elementary school children: Their interrelations and their role for test performance, Learn. Individ. Differ., vol. 29, pp. 141–149, 2014.

[36]

H. L. Swanson, Influence of metacognitive knowledge and aptitude on problem solving, J. Educ. Psychol., vol. 82, no. 2, pp. 306–314, 1990.

[37]

M. V. J. Veenman, R. Kok, and A. W. Blöte, The relation between intellectual and metacognitive skills in early adolescence, Instr. Sci., vol. 33, no. 3, pp. 193–211, 2005.

[38]

Z. R. Mevarech and C. Amrany, Immediate and delayed effects of meta-cognitive instruction on regulation of cognition and mathematics achievement, Metacognition Learn., vol. 3, no. 2, pp. 147–157, 2008.

[39]
R. J. Yinger, A study of teacher planning, Elem. Sch. J. , vol. 80, no. 3, pp. 107–127, 1980.
[40]
M. Hunter, Knowing, teaching and supervising, in Using What We Know About Reading, P. Hosford, ed. Alexandria, Egypt: Association for Supervision and Curriculum Development, 1984, pp. 169–203.
[41]

A. P. Johnson, It’s time for madeline hunter to go: A new look at lesson plan design, Action Teach. Educ., vol. 22, no. 1, pp. 72–78, 2000.

[42]
T. S. Farrell, Lesson planning, in Methodology in Language Teaching: An Anthology of Current Practice, J. C. Richards and W. A. Renandya, Eds. New York, NY, USA: Cambridge University Press, 2002, pp. 30–39.
[43]

G. Tonguç and B. O. Ozkara, Automatic recognition of student emotions from facial expressions during a lecture, Comput. Educ., vol. 148, p. 103797, 2020.

[44]
M. Ford, C. T. Baer, D. Xu, U. Yapanel, and S. Gray, The LENATM language environment analysis system: Audio specifications of the DLP-0121, Technical report, LENA Foundation, Boulder, CO, USA, 2008.
[45]

Z. Wang, K. Miller, and K. Cortina, Using the LENA in teacher training: Promoting student involvement through automated feedback, Unterrichtswissenschaft, vol. 4, pp. 290–305, 2013.

[46]

Z. Wang, X. Pan, K. F. Miller, and K. S. Cortina, Automatic classification of activities in classroom discourse, Comput. Educ., vol. 78, pp. 115–123, 2014.

[47]
D. Jiang, Y. Chen, and A. Garg, A hybrid method for overlapping speech detection in classroom environment, Comput. Appl. Eng. Educ., vol. 26, no. 1, pp. 171–180, 2018.
[48]

Y. Song, S. Lei, T. Hao, Z. Lan, and Y. Ding, Automatic classification of semantic content of classroom dialogue, J. Educ. Comput. Res., vol. 59, no. 3, pp. 496–521, 2021.

[49]
P. M. Moreno-Marcos, P. J. Muñoz-Merino, C. Alario-Hoyos, I. Estévez-Ayres, and C. Delgado Kloos, Analysing the predictive power for anticipating assignment grades in a massive open online course, Behav. Inf. Technol., vol. 37, nos. 10–11, pp. 1021–1036, 2018.
[50]
K. Mrhar, L. Benhiba, S. Bourekkache, and M. Abik, A Bayesian CNN-LSTM model for sentiment analysis in massive open online courses MOOCs, Int. J. Emerg. Technol. Learn. , vol. 16, no. 23, pp. 216–232, 2021.
[51]

S. Wachter, B. Mittelstadt, C. Russell, Counterfactual explanations without opening the black box: Automated decisions and the GDPR, SSRN Electron. J., vol. 31, no. 2, pp. 841–887, 2017.

[52]
A. Chatzimparmpas, R. M. Martins, I. Jusufi, and A. Kerren, A survey of surveys on the use of visualization for interpreting machine learning models, Inf. Vis. , vol. 19, no. 3, pp. 207–233, 2020.
[53]

G. Plumb, D. Molitor, A. S. Talwalkar, Model Agnostic Supervised Local Explanations, NIPS, vol. 31, pp. 2520–2529, 2018.

[54]
M. T. Ribeiro, S. Singh, C. Guestrin, Why should I trust you?: Explaining the predictions of any classifier, in Proc. 22nd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, San Francisco, CA, USA, 2016, pp. 1135–1144.
[55]
S. M. Lundberg, S. I. Lee, A unified approach to interpreting model predictions, NIPS, vol. 30, pp. 4768–4777, 2017.
[56]

S. Karlos, G. Kostopoulos, and S. Kotsiantis, Predicting and interpreting students’ grades in distance higher education through a semi-regression method, Appl. Sci., vol. 10, no. 23, p. 8413, 2020.

[57]
S. Kim, W. Kim, Y. Jang, S. Choi, H. Jung, and H. Kim, Student knowledge prediction for teacher-student interaction, in Proc. AAAI Conf. Artif. Intell. , vol. 35, no. 17, pp. 15560–15568, 2021.
[58]
J. Devlin, M. Chang, K. Lee, and K. Toutanova, BERT: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv: 1810.04805, 2018.
[59]

J. Hoth, G. Kaiser, A. Busse, M. Doehrmann, J. Koenig, and S. Blömeke, Professional competences of teachers for fostering creativity and supporting high-achieving students, ZDM Math. Educ., vol. 49, no. 1, pp. 107–120, 2017.

[60]
J. T. Guthrie and A. Wigfield, Engagement and motivation in reading, in Handbook of Reading Research, P. D. Barr, ed. New York, NY, USA: Lawrence Erlbaum Associates, 2000, pp. 403–422.
[61]

X. Xu, J. Wang, H. Peng, and R. Wu, Prediction of academic performance associated with Internet usage behaviors using machine learning algorithms, Comput. Hum. Behav., vol. 98, pp. 166–173, 2019.

[62]

C. F. Rodríguez-Hernández, M. Musso, E. Kyndt, and E. Cascallar, Artificial neural networks in academic performance prediction: Systematic implementation and predictor evaluation, Comput. Educ., vol. 2, p. 100018, 2021.

[63]

L. Zheng, M. Long, J. Niu, and L. Zhong, An automated group learning engagement analysis and feedback approach to promoting collaborative knowledge building, group performance, and socially shared regulation in CSCL, Int. J. Comput. Support. Collab. Learn., vol. 18, no. 1, pp. 101–133, 2023.

Journal of Social Computing
Pages 12-29
Cite this article:
Zhen Y, Luo J-D, Chen H. Prediction of Academic Performance of Students in Online Live Classroom Interactions —An Analysis Using Natural Language Processing and Deep Learning Methods. Journal of Social Computing, 2023, 4(1): 12-29. https://doi.org/10.23919/JSC.2023.0007

5023

Views

645

Downloads

8

Crossref

12

Scopus

Altmetrics

Received: 13 February 2023
Revised: 01 June 2023
Accepted: 06 June 2023
Published: 30 March 2023
© The author(s) 2023.

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return