Can Strategic Behaviour Facilitate Confusion Resolution? The Interplay Between Confusion and Metacognitive Strategies in Betty’s Brain
DOI:
https://doi.org/10.18608/jla.2021.7161Keywords:
confusion, confusion resolution, metacognitive strategy, cognitive disequilibriumAbstract
Confusion may benefit learning when it is resolved or partially resolved. Metacognitive strategies (MS) may help learners to resolve confusion when it occurs during learning and problem solving. This study examined the relationship between confusion and MS that students evoked in Betty’s Brain, a computer-based learning-by-modelling environment where elementary and middle school students learn science by building causal maps. Participants were sixth graders. Emotion data were collected from real-time observations by trained researchers. MS and task performance information were determined by analyzing the action logs. Pre- and post-tests were used to assess learning gains. The results revealed that the use of MS was a function of the state of student confusion. However, confusion resolution was not related to MS behaviour, and MS did not moderate the effect of confusion on student task performance in Betty’s Brain or on learning gains.
References
Arguel, A., Lockyer, L., Kennedy, G., Lodge, J. M., & Pachman, M. (2019). Seeking optimal confusion: A review on epistemic emotion management in interactive digital learning environments. Interactive Learning Environments, 27(2), 200–210. https://doi.org/10.1080/10494820.2018.1457544
Arguel, A., Lockyer, L., Lipp, O. V., Lodge, J. M., & Kennedy, G. (2017). Inside out: Detecting learners’ confusion to improve interactive digital learning environments. Journal of Educational Computing Research, 55(4), 526–551. https://doi.org/10.1177/0735633116674732
Azevedo, R., Behnagh, R., Duffy, M., Harley, J., & Trevors, G. (2012). Metacognition and self-regulated learning in student-centered leaning environments. In D. Jonassen & S. Land (Eds.), Theoretical foundations of student-centered learning environments (2nd ed., pp. 171–197). New York: Routledge. https://doi.org/10.4324/9780203813799
Baker, R. S., D’Mello, S. K., Rodrigo, M. M. T., & Graesser, A. C. (2010). Better to be frustrated than bored: The incidence, persistence, and impact of learners’ cognitive–affective states during interactions with three different computer-based learning environments. International Journal of Human–Computer Studies, 68(4), 223–241. https://doi.org/10.1016/j.ijhcs.2009.12.003
Biswas, G., Segedy, J. R., & Bunchongchit, K. (2016). From design to implementation to practice a learning by teaching system: Betty’s Brain. International Journal of Artificial Intelligence in Education, 26(1), 350–364. https://doi.org/10.1007/s40593-015-0057-9
Bosch, N., D’Mello, S., & Mills, C. (2013). What emotions do novices experience during their first computer programming learning session? In H. C. Lane, K. Yacef, J. Mostow, & P. Pavlik (Eds.), Proceedings of the 16th International Conference on Artificial Intelligence in Education (AIED ʼ13), 9–13 July 2013, Memphis, TN, USA (pp. 11–20). Lecture Notes in Computer Science, vol. 7926. Springer. https://doi.org/10.1007/978-3-642-39112-5_2
Botelho, A. F., Baker, R. S., Ocumpaugh, J., & Heffernan, N. T. (2018). Studying affect dynamics and chronometry using sensor-free detectors. In K. E. Boyer & M. Yudelson (Eds.), Proceedings of the 11th International Conference on Educational Data Mining (EDM2018), 16–20 July 2018, Buffalo, NY, USA (pp. 157–166). International Educational Data Mining Society. https://eric.ed.gov/?id=ED593106
Brunner, E., & Puri, M. L. (2001). Nonparametric methods in factorial designs. Statistical Papers, 42(1), 1–52. https://doi.org/10.1007/s003620000039
Calvo, R. A., & D’Mello, S. (2010). Affect detection: An interdisciplinary review of models, methods, and their applications. IEEE Transactions on Affective Computing, 1(1), 18–37. https://doi.org/10.1109/T-AFFC.2010.1
Caprara, G. V., Fida, R., Vecchione, M., Del Bove, G., Vecchio, G. M., Barbaranelli, C., ... Bandura, A. (2008). Longitudinal analysis of the role of perceived self-efficacy for self-regulated learning in academic continuance and achievement. Journal of Educational Psychology, 100(3), 525–534. https://doi.org/10.1037/0022-0663.100.3.525
Chevrier, M., Muis, K. R., Trevors, G. J., Pekrun, R., & Sinatra, G. M. (2019). Exploring the antecedents and consequences of epistemic emotions. Learning and Instruction, 63, 101209. https://doi.org/10.1016/j.learninstruc.2019.05.006
Cicchinelli, A., Veas, E., Pardo, A., Pammer-Schindler, V., Fessl, A., Barreiros, C., ... Lindstädt, S. (2018). Finding traces of self-regulated learning in activity streams. Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK ’18), 5–9 March 2018, Sydney, NSW, Australia (pp. 191–200). New York: ACM. https://doi.org/10.1145/3170358.3170381
Cooper, P. A. (1993). Paradigm shifts in designed instruction: From behaviorism to cognitivism to constructivism. Educational Technology, 33(5), 12–19. https://www.jstor.org/stable/44428049
Cowen, A., Sauter, D., Tracy, J. L., & Keltner, D. (2019). Mapping the passions: Toward a high-dimensional taxonomy of emotional experience and expression. Psychological Science in the Public Interest, 20(1), 69–90. https://doi.org/10.1177%2F1529100619850176
Craig, S., Graesser, A., Sullins, J., & Gholson, B. (2004). Affect and learning: An exploratory look into the role of affect in learning with AutoTutor. Journal of Educational Media, 29(3), 241–250. https://doi.org/10.1080/1358165042000283101
de Bruin, A. B. H., Thiede, K. W., Camp, G., & Redford, J. (2011). Generating keywords improves metacomprehension and self-regulation in elementary and middle school children. Journal of Experimental Child Psychology, 109(3), 294–310. https://doi.org/10.1016/j.jecp.2011.02.005
Dent, A. L., & Koenka, A. C. (2016). The relation between self-regulated learning and academic achievement across childhood and adolescence: A meta-analysis. Educational Psychology Review, 28(3), 425–474. https://doi.org/10.1007/s10648-015-9320-8
Di Leo, I., Muis, K. R., Singh, C. A., & Psaradellis, C. (2019). Curiosity… Confusion? Frustration! The role and sequencing of emotions during mathematics problem solving. Contemporary Educational Psychology, 58, 121–137. https://doi.org/10.1016/j.cedpsych.2019.03.001
D’Mello, S., & Graesser, A. (2012). Dynamics of affective states during complex learning. Learning and Instruction, 22(2), 145–157. https://doi.org/10.1016/j.learninstruc.2011.10.001
D’Mello, S. (2013). A selective meta-analysis on the relative incidence of discrete affective states during learning with technology. Journal of Educational Psychology, 105(4), 1082. https://doi.org/10.1037/a0032674
D’Mello, S., & Graesser, A. (2014a). Confusion. In R. Pekrun & L. Linnenbrink-Garcia (Eds.), International handbook of emotions in education (pp. 289–310). New York: Routledge. https://doi.org/10.4324/9780203148211.ch15
D’Mello, S., & Graesser, A. (2014b). Confusion and its dynamics during device comprehension with breakdown scenarios. Acta Psychologica, 151, 106–116. https://doi.org/10.1016/j.actpsy.2014.06.005
D’Mello, S., Lehman, B., Pekrun, R., & Graesser, A. (2014). Confusion can be beneficial for learning. Learning and Instruction, 29, 153–170. https://doi.org/10.1016/j.learninstruc.2012.05.003
Efklides, A. (2006). Metacognition and affect: What can metacognitive experiences tell us about the learning process? Educational Research Review, 1(1), 3–14. https://doi.org/https://doi.org/10.1016/j.edurev.2005.11.001
Efklides, A. (2008). Metacognition: Defining its facets and levels of functioning in relation to self-regulation and co-regulation. European Psychologist, 13(4), 277–287. https://doi.org/10.1027/1016-9040.13.4.277
Efklides, A. (2011). Interactions of metacognition with motivation and affect in self-regulated learning: The MASRL model. Educational Psychologist, 46(1), 6–25. https://doi.org/10.1080/00461520.2011.538645
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. American Psychologist, 34(10), 906. https://doi.org/10.1037/0003-066X.34.10.906
Hargrove, R. A., & Nietfeld, J. L. (2015). The impact of metacognitive instruction on creative problem solving. The Journal of Experimental Education, 83(3), 291–318. https://doi.org/10.1080/00220973.2013.876604
Jiang, Y., Bosch, N., Baker, R. S., Paquette, L., Ocumpaugh, J., Andres, J. M. A. L., Moore, A. L., & Biswas, G. (2018). Expert feature-engineering vs. deep neural networks: Which is better for sensor-free affect detection? In C. Penstein Rosé, R. Martínez-Maldonado, H. U. Hoppe, R. Luckin, M. Mavrikis, K. Porayska-Pomsta, B. McLaren, & B. du Boulay. (Eds.), Proceedings of the 19th International Conference on Artificial Intelligence in Education (AIED ʼ18), 27–30 June 2018, London, UK (pp. 198–211). Lecture Notes in Computer Science, vol. 10947. Springer. https://doi.org/10.1007/978-3-319-93843-1_15
Kinnebrew, J. S., Segedy, J. R., & Biswas, G. (2014). Analyzing the temporal evolution of students’ behaviours in open-ended learning environments. Metacognition and Learning, 9(2), 187–215. https://doi.org/10.1007/s11409-014-9112-4
Kinnebrew, J. S., Segedy, J. R., & Biswas, G. (2017). Integrating model-driven and data-driven techniques for analyzing learning behaviours in open-ended learning environments. IEEE Transactions on Learning Technologies, 10(2), 140–153. https://doi.org/10.1109/TLT.2015.2513387
Leelawong, K., & Biswas, G. (2008). Designing learning by teaching agents: The Betty’s Brain system. International Journal of Artificial Intelligence in Education, 18(3), 181–208. https://iaied.org/journal/997
Lehman, B., D’Mello, S., & Graesser, A. (2012). Confusion and complex learning during interactions with computer learning environments. The Internet and Higher Education, 15(3), 184–194. https://doi.org/10.1016/j.iheduc.2012.01.002
Lehman, B., D’Mello, S., & Graesser, A. (2013). Who benefits from confusion induction during learning? An individual differences cluster analysis. In H. C. Lane, K. Yacef, J. Mostow, & P. Pavlik (Eds.), Proceedings of the 16th International Conference on Artificial Intelligence in Education (AIED ʼ13), 9–13 July 2013, Memphis, TN, USA (pp. 51–60). Lecture Notes in Computer Science, vol. 7926. Springer. https://doi.org/10.1007/978-3-642-39112-5_6
Lehman, B., & Graesser, A. (2014). Impact of agent role on confusion induction and learning. In S. Trausan-Matu, K. E. Boyer, M. Crosby, & K. Panourgia (Eds.), Proceedings of the 12th International Conference on Intelligent Tutoring Systems (ITS 2014), 5–9 June 2014, Honolulu, HI, USA (pp. 45–54). Lecture Notes in Computer Science, vol. 8474. New York: Springer. https://doi.org/10.1007/978-3-319-07221-0_6
Lehman, B., & Graesser, A. (2015). To resolve or not to resolve? That is the big question about confusion. In C. Conati, N. Heffernan, A. Mitrovic, & M. Verdejo (Eds.), Proceedings of the 17th International Conference on Artificial Intelligence in Education (AIED ʼ15), 22–26 June 2015, Madrid, Spain (pp. 216–225). Lecture Notes in Computer Science, vol. 9112. Springer. https://doi.org/10.1007/978-3-319-19773-9_22
Liu, Z., Pataranutaporn, V., Ocumpaugh, J., & Baker, R. S. (2013). Sequences of frustration and confusion, and learning. In S. K. D’Mello, R. A. Calvo, & A. Olney (Eds.), Proceedings of the 6th International Conference on Educational Data Mining (EDM 2013), 6–9 July 2013, Memphis, TN, USA (pp. 114–120). International Educational Data Mining Society. http://educationaldatamining.org/EDM2013/proceedings/paper_34.pdf
Mair, P., & Wilcox, R. (2020). Robust statistical methods in R using the WRS2 package. Behaviour Research Methods, 52, 464–488. https://doi.org/10.3758/s13428-019-01246-w
Mandler, G. (1990). Interruption (discrepancy) theory: Review and extensions. In S. Fisher & C. L. Cooper (Eds.), On the move: The psychology of change and transition (pp. 13–32). Chichester: Wiley.
Meany Daboul, M. G., Roscoe, E. M., Bourret, J. C., & Ahearn, W. H. (2007). A comparison of momentary time sampling and partial‐interval recording for evaluating functional relations. Journal of Applied Behaviour Analysis, 40(3), 501–514. https://doi.org/10.1901/jaba.2007.40-501
Muis, K. R., Chevrier, M., & Singh, C. A. (2018). The role of epistemic emotions in personal epistemology and self-regulated learning. Educational Psychologist, 53(3), 165–184. https://doi.org/10.1080/00461520.2017.1421465
Muis, K. R., Pekrun, R., Sinatra, G. M., Azevedo, R., Trevors, G., Meier, E., ... Heddy, B. C. (2015). The curious case of climate change: Testing a theoretical model of epistemic beliefs, epistemic emotions, and complex learning. Learning and Instruction, 39, 168–183. https://doi.org/10.1016/j.learninstruc.2015.06.003
Muis, K. R., Psaradellis, C., Lajoie, S. P., Di Leo, I., & Chevrier, M. (2015). The role of epistemic emotions in mathematics problem solving. Contemporary Educational Psychology, 42, 172–185. https://doi.org/10.1016/j.cedpsych.2015.06.003
Munzar, B., Muis, K. R., Denton, C. A., & Losenno, K. (2020). Elementary students’ cognitive and affective responses to impasses during mathematics problem solving. Journal of Educational Psychology, 113(1), 104–124. https://doi.org/http://doi.org/10.1037/edu0000460
Nelson, T. O., & Narens, L. (1990). Metamemory: A theoretical framework and new findings. Psychology of Learning and Motivation, 26, 125–173. https://doi.org/10.1016/S0079-7421(08)60053-5
Noguchi, K., Gel, Y. R., Brunner, E., & Konietschke, F. (2012). nparLD: An R software package for the nonparametric analysis of longitudinal data in factorial experiments. Journal of Statistical Software, 50(12). https://doi.org/10.18637/jss.v050.i12
Ocumpaugh, J., Baker, R. S., & Rodrigo, M. M. T. B. (2015). Baker Rodrigo Ocumpaugh monitoring protocol (BROMP) 2.0 technical and training manual. New York, NY/Manila, Philippines: Teachers College, Columbia University and Ateneo Laboratory for the Learning Sciences. https://www.upenn.edu/learninganalytics/ryanbaker/BROMP.pdf
Ocumpaugh, J., Baker, R. S., Rodrigo, M. M., Salvi, A., Van Velsen, M., Aghababyan, A., ... Martin, T. (2015). HART: The human affect recording tool. Proceedings of the 33rd Annual International Conference on the Design of Communication (SIGDOC ’15), 16–17 July 2015, Limerick, Ireland (Article 24). New York: ACM. https://doi.org/10.1145/2775441.2775480
Ohtani, K., & Hisasaka, T. (2018). Beyond intelligence: A meta-analytic review of the relationship among metacognition, intelligence, and academic performance. Metacognition and Learning, 13(2), 179–212. https://doi.org/10.1007/s11409-018-9183-8
Pekrun, R., & Stephens, E. J. (2012). Academic emotions. In K. Harris, S. Graham, T. Urdan, S. Graham, & J. Royer (Eds.), APA educational psychology handbook: Vol. 2. Individual differences and cultural and contextual factors (pp. 3–31). Washington, DC: American Psychological Association. https://doi.org/10.1037/13274-000
Pintrich, P. R., Smith, D. A., Garcia, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement, 53(3), 801–813. https://doi.org/10.1177/0013164493053003024
Planalp, S., DeFrancisco, V. L., & Rutherford, D. (1996). Varieties of cues to emotion in naturally occurring settings. Cognition and Emotion, 10(2), 137–153. https://doi.org/10.1080/026999396380303
Rodrigo, M. M. T., Baker, R. S. J. D., D’Mello, S., Gonzalez, M. C. T., Lagud, M. C. V., Lim, S. A. L.,... Viehland, N. J. B. (2008). Comparing learners’ affect while using an Intelligent Tutoring System and a simulation problem solving game. In B. Woolf, E. Aïmeur, R. Nkambou, & S. Lajoie (Eds.), Proceedings of the 9th International Conference on Intelligent Tutoring Systems (ITS 2008), 23–27 June 2008, Montreal, QC, Canada (pp. 40–49). Lecture Notes in Computer Science, vol. 5091. Springer. https://doi.org/10.1007/978-3-540-69132-7_9
Segedy, J. R., Biswas, G., & Sulcer, B. (2014). A model-based behaviour analysis approach for open-ended environments. Journal of Educational Technology & Society, 17(1). https://doi.org/https://www.jstor.org/stable/26896707
Segedy, J. R., Kinnebrew, J. S., & Biswas, G. (2015). Using coherence analysis to characterize self-regulated learning behaviours in open-ended learning environments. Journal of Learning Analytics, 2(1), 13–48. https://doi.org/10.18608/jla.2015.21.3
Silvia, P. J. (2010). Confusion and interest: The role of knowledge emotions in aesthetic experience. Psychology of Aesthetics, Creativity, and the Arts, 4(2), 75–80. https://doi.org/10.1037/a0017081
van der Stel, M., & Veenman, M. V. (2014). Metacognitive skills and intellectual ability of young adolescents: A longitudinal study from a developmental perspective. European Journal of Psychology of Education, 29(1), 117–137. https://doi.org/10.1007/s10212-013-0190-5
Vogl, E., Pekrun, R., Murayama, K., Loderer, K., & Schubert, S. (2019). Surprise, curiosity, and confusion promote knowledge exploration: Evidence for robust effects of epistemic emotions. Frontiers in Psychology, 10, Article 2474. https://doi.org/10.3389/fpsyg.2019.02474
Vogl, E., Pekrun, R., Murayama, K., & Loderer, K. (2020). Surprised–curious–confused: Epistemic emotions and knowledge exploration. Emotion, 20(4), 625–641. https://doi.org/10.1037/emo0000578
Wang, M. C., Haertel, G. D., & Walberg, H. J. (1990). What influences learning? A content analysis of review literature. The Journal of Educational Research, 84(1), 30–43. https://www.jstor.org/stable/40539680
Wilcox, R. R. (2011). Introduction to robust estimation and hypothesis testing (3rd ed.). Oxford, UK: Elsevier.
Wilcox, R. R., & Tian, T. S. (2011). Measuring effect size: A robust heteroscedastic approach for two or more groups. Journal of Applied Statistics, 38(7), 1359–1368. https://doi.org/10.1080/02664763.2010.498507
Yang, D., Kraut, R., & Rose, C. P. (2016). Exploring the effect of student confusion in massive open online courses. Journal of Educational Data Mining, 8(1), 52–83. https://doi.org/10.5281/zenodo.3554605
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2021 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
