Prior studies have shown the importance of classroom dialogue in academic performance, through which knowledge construction and social interaction among students take place. However, most of them were based on small scale or qualitative data, and few has explored the availability and potential of big data collected from online classrooms. To address this issue, this paper analyzes dialogues in live classrooms of a large online learning platform in China based on natural language processing techniques. The features of interactive types and emotional expression are extracted from classroom dialogues. We then develop neural network models based on these features to predict high- and low-academic performing students, and employ interpretable AI (artificial intelligence) techniques to determine the most important predictors in the prediction models. In both STEM (science, technology, engineering, mathematics) and non-STEM courses, it is found that high-performing students consistently exhibit more positive emotion, cognition and off-topic dialogues in all stages of the lesson than low-performing students. However, while the metacognitive dialogue illustrates its importance in non-STEM courses, this effect cannot be found in STEM courses. While high-performing students in non-STEM courses show negative emotion in the last stage of lessons, STEM students show positive emotion.
J. E. Knowles, Of needles and haystacks: Building an accurate statewide dropout early warning system in wisconsin, J. Educ. Data Min., vol. 7, no. 3, pp. 18–67, 2015.
S. Lee and J. Y. Chung, The machine learning-based dropout early warning system for improving the performance of dropout prediction, Appl. Sci., vol. 9, no. 15, p. 3093, 2019.
L. Yan, A. Whitelock-Wainwright, Q. Guan, G. Wen, D. Gašević, and G. Chen, Students’ experience of online learning during the COVID-19 pandemic: A province-wide survey study, Br. J. Educ. Technol., vol. 52, no. 5, pp. 2038–2057, 2021.
S. Hennessy, S. Rojas-Drummond, R. Higham, A. M. Márquez, F. Maine, R. M. Ríos, R. García-Carrión, O. Torreblanca, and M. J. Barrera, Developing a coding scheme for analysing classroom dialogue across educational contexts, Learn. Cult. Soc. Interact., vol. 9, pp. 16–44, 2016.
M. Zembylas, Adult learners’ emotions in online learning, Distance Educ., vol. 29, no. 1, pp. 71–87, 2008.
W. Zou, X. Hu, Z. Pan, C. Li, Y. Cai, and M. Liu, Exploring the relationship between social presence and learners’ prestige in MOOC discussion forums using automated content analysis and social network analysis, Comput. Hum. Behav., vol. 115, p. 106582, 2021.
C. Howe and M. Abedin, Classroom dialogue: A systematic review across four decades of research, Camb. J. Educ., vol. 43, no. 3, pp. 325–356, 2013.
H. Muhonen, E. Pakarinen, A. M. Poikkeus, M. K. Lerkkanen, and H. Rasku-Puttonen, Quality of educational dialogue and association with students’ academic performance, Learn. Instr., vol. 55, pp. 67–79, 2018.
G. Zhu, W. Xing, S. Costa, M. Scardamalia, and B. Pei, Exploring emotional and cognitive dynamics of Knowledge Building in grades 1 and 2, User Model. User Adapt. Interact., vol. 29, no. 4, pp. 789–820, 2019.
K. R. Scherer, What are emotions? And how can they be measured, Soc. Sci. Inf., vol. 44, no. 4, pp. 695–729, 2005.
A. Bakhtiar, E. A. Webster, and A. F. Hadwin, Regulation and socio-emotional interactions in a positive and a negative group climate, Metacognition Learn., vol. 13, no. 1, pp. 57–90, 2018.
M. Cleveland-Innes and P. Campbell, Emotional presence, learning, and the online learning environment, Int. Rev. Res. Open Distrib. Learn., vol. 13, no. 4, p. 269, 2012.
A. R. Damasio, Descartes’ error and the future of human life, Sci. Am., vol. 271, no. 4, p. 144, 1994.
L. Yorks and E. Kasl, Toward a theory and practice for whole-person learning: Reconceptualizing experience and the role of affect, Adult Educ. Q., vol. 52, no. 3, pp. 176–192, 2002.
R. Pekrun, S. Lichtenfeld, H. W. Marsh, K. Murayama, and T. Goetz, Achievement emotions and academic performance: Longitudinal models of reciprocal effects, Child Dev., vol. 88, no. 5, pp. 1653–1670, 2017.
R. S. J. D. Baker, S. K. D'Mello, M. M. T. Rodrigo, and A. C. Graesser, Better to be frustrated than bored: The incidence, persistence, and impact of learners’ cognitive-affective states during interactions with three different computer-based learning environments, Int. J. Hum. Comput. Stud., vol. 68, no. 4, pp. 223–241, 2010.
S. D’Mello, B. Lehman, R. Pekrun, and A. Graesser, Confusion can be beneficial for learning, Learn. Instr., vol. 29, pp. 153–170, 2014.
L. Zheng, J. Niu, and L. Zhong, Effects of a learning analytics-based real-time feedback approach on knowledge elaboration, knowledge convergence, interactive relationships and group performance in CSCL, Br. J. Educ. Technol., vol. 53, no. 1, pp. 130–149, 2022.
L. Przybilla, K. Klinker, M. Kauschinger, and H. Krcmar, Stray off topic to stay on topic: Preserving interaction and team morale in a highly collaborative course while at a distance, Commun. Assoc. Inf. Syst., vol. 48, no. 1, pp. 177–184, 2021.
J. H. Flavell, Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry, Am. Psychol., vol. 34, no. 10, pp. 906–911, 1979.
M. E. Martinez, What is metacognition, Phi Delta Kappan, vol. 87, no. 9, pp. 696–699, 2006.
C. D. Zepeda, C. O. Hlutkowsky, A. C. Partika, and T. J. Nokes-Malach, Identifying teachers’ supports of metacognition through classroom talk and its relation to growth in conceptual learning, J. Educ. Psychol., vol. 111, no. 3, pp. 522–541, 2019.
P. H. Winne and J. C. Nesbit, The psychology of academic achievement, Annu. Rev. Psychol., vol. 61, pp. 653–678, 2010.
A. Cadamuro, E. Bisagno, G. A. Di Bernardo, L. Vezzali, and A. Versari, Making the school Smart: the interactive whiteboard against disparities in children stemming from low metacognitive skills, J. E-Learn. Knowl. Soc., vol. 16, no. 1, pp. 33–43, 2020.
C. M. Roebers, S. S. Krebs, and T. Roderer, Metacognitive monitoring and control in elementary school children: Their interrelations and their role for test performance, Learn. Individ. Differ., vol. 29, pp. 141–149, 2014.
H. L. Swanson, Influence of metacognitive knowledge and aptitude on problem solving, J. Educ. Psychol., vol. 82, no. 2, pp. 306–314, 1990.
M. V. J. Veenman, R. Kok, and A. W. Blöte, The relation between intellectual and metacognitive skills in early adolescence, Instr. Sci., vol. 33, no. 3, pp. 193–211, 2005.
Z. R. Mevarech and C. Amrany, Immediate and delayed effects of meta-cognitive instruction on regulation of cognition and mathematics achievement, Metacognition Learn., vol. 3, no. 2, pp. 147–157, 2008.
A. P. Johnson, It’s time for madeline hunter to go: A new look at lesson plan design, Action Teach. Educ., vol. 22, no. 1, pp. 72–78, 2000.
G. Tonguç and B. O. Ozkara, Automatic recognition of student emotions from facial expressions during a lecture, Comput. Educ., vol. 148, p. 103797, 2020.
Z. Wang, K. Miller, and K. Cortina, Using the LENA in teacher training: Promoting student involvement through automated feedback, Unterrichtswissenschaft, vol. 4, pp. 290–305, 2013.
Z. Wang, X. Pan, K. F. Miller, and K. S. Cortina, Automatic classification of activities in classroom discourse, Comput. Educ., vol. 78, pp. 115–123, 2014.
Y. Song, S. Lei, T. Hao, Z. Lan, and Y. Ding, Automatic classification of semantic content of classroom dialogue, J. Educ. Comput. Res., vol. 59, no. 3, pp. 496–521, 2021.
S. Wachter, B. Mittelstadt, C. Russell, Counterfactual explanations without opening the black box: Automated decisions and the GDPR, SSRN Electron. J., vol. 31, no. 2, pp. 841–887, 2017.
G. Plumb, D. Molitor, A. S. Talwalkar, Model Agnostic Supervised Local Explanations, NIPS, vol. 31, pp. 2520–2529, 2018.
S. Karlos, G. Kostopoulos, and S. Kotsiantis, Predicting and interpreting students’ grades in distance higher education through a semi-regression method, Appl. Sci., vol. 10, no. 23, p. 8413, 2020.
J. Hoth, G. Kaiser, A. Busse, M. Doehrmann, J. Koenig, and S. Blömeke, Professional competences of teachers for fostering creativity and supporting high-achieving students, ZDM Math. Educ., vol. 49, no. 1, pp. 107–120, 2017.
X. Xu, J. Wang, H. Peng, and R. Wu, Prediction of academic performance associated with Internet usage behaviors using machine learning algorithms, Comput. Hum. Behav., vol. 98, pp. 166–173, 2019.
C. F. Rodríguez-Hernández, M. Musso, E. Kyndt, and E. Cascallar, Artificial neural networks in academic performance prediction: Systematic implementation and predictor evaluation, Comput. Educ., vol. 2, p. 100018, 2021.
L. Zheng, M. Long, J. Niu, and L. Zhong, An automated group learning engagement analysis and feedback approach to promoting collaborative knowledge building, group performance, and socially shared regulation in CSCL, Int. J. Comput. Support. Collab. Learn., vol. 18, no. 1, pp. 101–133, 2023.
This work was supported by the Center for Social Network Research of Tsinghua University, Tsinghua’s Research Project (No. 2016THZWYY03), and the Project of Tencent Social Research Center (No. 20162001703).
The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).