Learners’ Linguistic Alignment and Physiological Synchrony
Identifying Trigger Events that Invite Socially Shared Regulation of Learning
DOI:
https://doi.org/10.18608/jla.2024.8287Keywords:
collaboration analytics, collaborative learning, linguistic alignment, multimodal learning analytics, physiological synchrony, self-regulated learning (SRL), socially shared regulation of learning (SSRL), research paperAbstract
The theory of socially shared regulation of learning (SSRL) suggests that successful collaborative groups can identify and respond to trigger events stemming from cognitive or emotional obstacles in learning. Thus, to develop real-time support for SSRL, novel metrics are needed to identify different types of trigger events that invite SSRL. Our aim was to apply two metrics derived from different data streams to study how trigger events for SSRL shaped group linguistic alignment (based on audio data) and physiological synchrony (based on electrodermal activity data). The data came from six groups of students (N = 18) as they worked face-to-face on a collaborative learning task with one cognitive and two emotional trigger events. We found that the cognitive trigger event increased linguistic alignment in task-description words and led to physiological out-of-synchrony. The emotional trigger events decreased out-of-synchrony and increased high-arousal synchrony at the physiological level but did not affect linguistic alignment. Therefore, different metrics for studying markers and responses to different types of trigger events are needed, suggesting the necessity for multimodal learning analytics to support collaborative learning.
References
Ates, H. C., Nguyen, P. Q., Gonzalez-Macia, L., Morales-Narváez, E., Güder, F., Collins, J. J., & Dincer, C. (2022). End-to-end design of wearable sensors. Nature Reviews Materials, 7(11), 887–907. https://doi.org/10.1038/s41578-022-00460-x
Benedek, M., & Kaernbach, C. (2010). A continuous measure of phasic electrodermal activity. Journal of Neuroscience Methods, 190(1), 80–91. https://doi.org/10.1016/j.jneumeth.2010.04.028
Blikstein, P., Gomes, J. S., Akiba, H. T., & Schneider, B. (2017). The effect of highly scaffolded versus general instruction on students’ exploratory behavior and arousal. Technology, Knowledge and Learning, 22(1), 105–128. https://doi.org/10.1007/s10758-016-9291-y
Branigan, H. P., Pickering, M. J., & Cleland, A. A. (2000). Syntactic co-ordination in dialogue. Cognition, 75(2), B13–B25. https://doi.org/10.1016/S0010-0277(99)00081-5
Brennan, S. E., & Clark, H. H. (1996). Conceptual pacts and lexical choice in conversation. Journal of Experimental Psychology, 22(6), 1482–1493. https://doi.org/10.1037/0278-7393.22.6.1482
Buseyne, S., Rajagopal, K., Danquigny, T., Depaepe, F., Heutte, J., & Raes, A. (2024). Assessing verbal interaction of adult learners in computer-supported collaborative problem solving. British Journal of Educational Technology, 55(4), 1465–1485. https://doi.org/10.1111/bjet.13391
Dang, B., Nguyen, A., & Järvelä, S. (2023). Clustering deliberation sequences through regulatory triggers in collaborative learning. 2023 IEEE International Conference on Advanced Learning Technologies (ICALT 2023), 10–13 July 2023, Orem, UT, USA (pp. 158–160). https://doi.org/10.1109/ICALT58122.2023.00052
Dawson, M. E., Schell, A. M., & Filion, D. L. (2016). The electrodermal system. In J. T. Cacioppo, L. G. Tassinary, & G. G. Berntson (Eds.), Handbook of psychophysiology (4th ed., pp. 217–243). Cambridge University Press. https://doi.org/10.1017/9781107415782.010
Dich, Y., Reilly, J., & Schneider, B. (2018). Using physiological synchrony as an indicator of collaboration quality, task performance and learning. Proceedings of the 19th International Conference on Artificial Intelligence in Education (AIED 2018), 27–30 July 2018, London, United Kingdom (pp. 98–110). Springer Cham. https://doi.org/10.1007/978-3-319-93843-1_8
Dideriksen, C., Christiansen, M. H., Tylén, K., Dingemanse, M., & Fusaroli, R. (2023). Quantifying the interplay of conversational devices in building mutual understanding. Journal of Experimental Psychology, 152(3), 864–889. https://doi.org/10.1037/xge0001301
Dindar, M., Järvelä, S., & Haataja, E. (2020). What does physiological synchrony reveal about metacognitive experiences and group performance? British Journal of Educational Technology, 51(5), 1577–1596. https://doi.org/10.1111/bjet.12981
Dindar, M., Järvelä, S., Nguyen, A., Haataja, E., & Çini, A. (2022). Detecting shared physiological arousal events in collaborative problem solving. Contemporary Educational Psychology, 69, 102050. https://doi.org/10.1016/j.cedpsych.2022.102050
Dubuisson Duplessis, G., Clavel, C., & Landragin, F. (2017). Automatic measures to characterise verbal alignment in human-agent interaction. Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue (SIGDIAL 2017), 15–17 August 2017, Saarbrücken, Germany (pp. 71–81). Association for Computational Linguistics. https://doi.org/10.18653/v1/W17-5510
Emara, M., Hutchins, N., Grover, S., Snyder, C., & Biswas, G. (2021). Examining student regulation of collaborative, computational, problem-solving processes in open-ended learning environments. Journal of Learning Analytics, 8(1), 49–74. https://doi.org/10.18608/jla.2021.7230
Febriantoro, W., Gauthier, A., & Cukurova, M. (2023). The promise of physiological data in collaborative learning: A systematic literature review. Proceedings of the 18th European Conference on Technology Enhanced Learning (EC-TEL 2023), 4–8 September 2023, Aveiro, Portual (pp. 75–88). Springer Cham. https://doi.org/10.1007/978-3-031-42682-7_6
Friedberg, H., Litman, D., & Paletz, S. B. F. (2012). Lexical entrainment and success in student engineering groups. 2012 IEEE Spoken Language Technology Workshop (SLT 2012), 2–5 December 2012, Miami, FL, USA (pp. 404–409). IEEE. https://doi.org/10.1109/SLT.2012.6424258
Galy, E., Cariou, M., & Mélan, C. (2012). What is the relationship between mental workload factors and cognitive load types? International Journal of Psychophysiology, 83(3), 269–275. https://doi.org/10.1016/j.ijpsycho.2011.09.023
Giannakos, M., & Cukurova, M. (2023). The role of learning theory in multimodal learning analytics. British Journal of Educational Technology, 54(5), 1246–1267. https://doi.org/10.1111/bjet.13320
Giles, H., Coupland, J., & Coupland, N. (Eds.). (1991). Contexts of accommodation: Developments in applied sociolinguistics. Cambridge University Press. https://doi.org/10.1017/CBO9780511663673
Haataja, E., Malmberg, J., Dindar, M., & Järvelä, S. (2022). The pivotal role of monitoring for collaborative problem solving seen in interaction, performance, and interpersonal physiology. Metacognition and Learning, 17(1), 241–268. https://doi.org/10.1007/s11409-021-09279-3
Hadwin, A., Järvelä, S., & Miller, M. (2017). Self-regulation, co-regulation, and shared regulation in collaborative learning environments. In D. H. Schunk & J. A. Greene (Eds.), Handbook of self-regulation of learning and performance (2nd ed., pp. 83–106). Routledge. https://doi.org/10.4324/9781315697048-6
Hayashi, Y. (2023). Modeling synchronization for detecting collaborative learning process using a pedagogical conversational agent: Investigation using recurrent indicators of gaze, language, and facial expression. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-023-00381-y
Hollink, V., Kamps, J., Monz, C., & de Rijke, M. (2004). Monolingual document retrieval for European languages. Information Retrieval, 7(1), 33–52. https://doi.org/10.1023/B:INRT.0000009439.19151.4c
Järvelä, S., & Hadwin, A. F. (2013). New frontiers: Regulating learning in CSCL. Educational Psychologist, 48(1), 25–39. https://doi.org/10.1080/00461520.2012.748006
Järvelä, S., Nguyen, A., & Hadwin, A. (2023). Human and artificial intelligence collaboration for socially shared regulation in learning. British Journal of Educational Technology, 54(5), 1057–1076. https://doi.org/10.1111/bjet.13325
Järvelä, S., Nguyen, A., Vuorenmaa, E., Malmberg, J., & Järvenoja, H. (2023). Predicting regulatory activities for socially shared regulation to optimize collaborative learning. Computers in Human Behavior, 144, 107737. https://doi.org/10.1016/j.chb.2023.107737
Lämsä, J., Uribe, P., Jiménez, A., Caballero, D., Hämäläinen, R., & Araya, R. (2021). Deep networks for collaboration analytics: Promoting automatic analysis of face-to-face interaction in the context of inquiry-based learning. Journal of Learning Analytics, 8(1), 113–125. https://doi.org/10.18608/jla.2021.7118
Li, X., Hu, W., Li, Y., & Mao, Z. (2023). Exploring what synchronized physiological arousal can reveal about the social regulatory process in a collaborative argumentation activity. Frontiers in Psychology, 13, 1042970. https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2022.1042970
Liu, L., Xia, Z., Zhang, X., Peng, J., Feng, X., & Zhao, G. (2024). Information-enhanced network for noncontact heart rate estimation from facial videos. IEEE Transactions on Circuits and Systems for Video Technology, 34(4), 2136–2150. https://doi.org/10.1109/TCSVT.2023.3301962
Liu, Y., Wang, T., Wang, K., & Zhang, Y. (2021). Collaborative learning quality classification through physiological synchrony recorded by wearable biosensors. Frontiers in Psychology, 12, 674369. https://www.frontiersin.org/articles/10.3389/fpsyg.2021.674369
Malmberg, J., Haataja, E., & Järvelä, S. (2022). Exploring the connection between task difficulty, task perceptions, physiological arousal and learning outcomes in collaborative learning situations. Metacognition and Learning, 17(3), 793–811. https://doi.org/10.1007/s11409-022-09320-z
Malmberg, J., Haataja, E., Törmänen, T., Järvenoja, H., Zabolotna, K., & Järvelä, S. (2023). Multimodal measures characterizing collaborative groups’ interaction and engagement in learning. In V. Kovanovic, R. Azevedo, D. C. Gibson, & D. lfenthaler (Eds.), Unobtrusive observations of learning in digital environments: Examining behavior, cognition, emotion, metacognition and social processes using learning analytics (pp. 197–216). Springer Cham. https://doi.org/10.1007/978-3-031-30992-2_12
Malmberg, J., Järvelä, S., Holappa, J., Haataja, E., Huang, X., & Siipo, A. (2019). Going beyond what is visible: What multichannel data can reveal about interaction in the context of collaborative learning? Computers in Human Behavior, 96, 235–245. https://doi.org/10.1016/j.chb.2018.06.030
Martinez-Maldonado, R., Gašević, D., Echeverria, V., Fernandez Nieto, G., Swiecki, Z., & Buckingham Shum, S. (2021). What do you mean by collaboration analytics? A conceptual model. Journal of Learning Analytics, 8(1), 126–153. https://doi.org/10.18608/jla.2021.7227
Miller, M., & Hadwin, A. (2015). Scripting and awareness tools for regulating collaborative learning: Changing the landscape of support in CSCL. Computers in Human Behavior, 52, 573–588. https://doi.org/10.1016/j.chb.2015.01.050
Molenaar, I., de Mooij, S., Azevedo, R., Bannert, M., Järvelä, S., & Gašević, D. (2023). Measuring self-regulated learning and the role of AI: Five years of research using multimodal multichannel data. Computers in Human Behavior, 139, 107540. https://doi.org/10.1016/j.chb.2022.107540
Mønster, D., Håkonsson, D. D., Eskildsen, J. K., & Wallot, S. (2016). Physiological evidence of interpersonal dynamics in a cooperative production task. Physiology & Behavior, 156, 24–34. https://doi.org/10.1016/j.physbeh.2016.01.004
Mooney, C. Z., & Duval, R. D. (1993). Bootstrapping: A nonparametric approach to statistical inference. Sage. https://doi.org/10.4135/9781412983532
Näykki, P., Isohätälä, J., Järvelä, S., Pöysä-Tarhonen, J., & Häkkinen, P. (2017). Facilitating socio-cognitive and socio-emotional monitoring in collaborative learning with a regulation macro script: An exploratory study. International Journal of Computer-Supported Collaborative Learning, 12(3), 251–279. https://doi.org/10.1007/s11412-017-9259-5
Palumbo, R. V., Marraccini, M. E., Weyandt, L. L., Wilder-Smith, O., McGee, H. A., Liu, S., & Goodwin, M. S. (2017). Interpersonal autonomic physiology: A systematic review of the literature. Personality and Social Psychology Review, 21(2), 99–141. https://doi.org/10.1177/1088868316628405
Pickering, M. J., & Garrod, S. (2004). Toward a mechanistic psychology of dialogue. Behavioral and Brain Sciences, 27(2), 169–190. https://doi.org/10.1017/S0140525X04000056
Quadt, L., Critchley, H., & Nagai, Y. (2022). Cognition, emotion, and the central autonomic network. Autonomic Neuroscience, 238, 102948. https://doi.org/10.1016/j.autneu.2022.102948
Rahimi, Z., Kumar, A., Litman, D., Paletz, S., & Yu, M. (2017). Entrainment in multi-party spoken dialogues at multiple linguistic levels. Interspeech 2017, 20–24 August 2017, Stockholm, Sweden (pp. 1696–1700). International Speech Communication Association. https://doi.org/10.21437/Interspeech.2017-1568
Roos, A.-L., Goetz, T., Voracek, M., Krannich, M., Bieg, M., Jarrell, A., & Pekrun, R. (2021). Test anxiety and physiological arousal: A systematic review and meta-analysis. Educational Psychology Review, 33(2), 579–618. https://doi.org/10.1007/s10648-020-09543-z
Schneider, B., Dich, Y., & Radu, I. (2020). Unpacking the relationship between existing and new measures of physiological synchrony and collaborative learning: A mixed methods study. International Journal of Computer-Supported Collaborative Learning, 15(1), 89–113. https://doi.org/10.1007/s11412-020-09318-2
Schneider, B., Sung, G., Chng, E., & Yang, S. (2021). How can high-frequency sensors capture collaboration? A review of the empirical links between multimodal metrics and collaborative constructs. Sensors, 21(24), 8185. https://doi.org/10.3390/s21248185
Sharma, K., Pappas, I., Papavlasopoulou, S., & Giannakos, M. (2019). Towards automatic and pervasive physiological sensing of collaborative learning. Proceedings of the 13th International Conference on Computer Supported Collaborative Learning 2019 (CSCL 2019), 17–21 June 2019, Lyon, France (pp. 684–687). International Society of the Learning Sciences. https://repository.isls.org//handle/1/1652
Sinclair, A. J., & Schneider, B. (2021). Linguistic and gestural coordination: Do learners converge in collaborative dialogue? Proceedings of the 14th International Conference on Educational Data Mining (EDM21), 29 June–2 July 2021, Paris, France (pp. 431–438). International Educational Data Mining Society. https://eric.ed.gov/?id=ED615547
Sobocinski, M., Järvelä, S., Malmberg, J., Dindar, M., Isosalo, A., & Noponen, K. (2020). How does monitoring set the stage for adaptive regulation or maladaptive behavior in collaborative learning? Metacognition and Learning, 15(2), 99–127. https://doi.org/10.1007/s11409-020-09224-w
Society for Psychophysiological Research Ad Hoc Committee on Electrodermal Measures. (2012). Publication recommendations for electrodermal measurements. Psychophysiology, 49(8), 1017–1034. https://doi.org/10.1111/j.1469-8986.2012.01384.x
Sung, G., Bhinder, H., Feng, T., & Schneider, B. (2023). Stressed or engaged? Addressing the mixed significance of physiological activity during constructivist learning. Computers & Education, 199, 104784. https://doi.org/10.1016/j.compedu.2023.104784
Suraworachet, W., Seon, J., & Cukurova, M. (2024). Predicting challenge moments from students’ discourse: A comparison of GPT-4 to two traditional natural language processing approaches. Proceedings of the 14th Learning Analytics and Knowledge Conference (LAK ’24), 18–22 March 2024, Kyoto, Japan (pp. 473–485). https://doi.org/10.1145/3636555.3636905
Törmänen, T., Järvenoja, H., Saqr, M., Malmberg, J., & Järvelä, S. (2023). Affective states and regulation of learning during socio-emotional interactions in secondary school collaborative groups. British Journal of Educational Psychology, 93(S1), 48–70. https://doi.org/10.1111/bjep.12525
Vuorenmaa, E., Järvelä, S., Dindar, M., & Järvenoja, H. (2023). Sequential patterns in social interaction states for regulation in collaborative learning. Small Group Research, 54(4), 512–550. https://doi.org/10.1177/10464964221137524
Wang, Z., Ng, J. T. D., Que, Y., & Hu, X. (2024). Unveiling synchrony of learners’ multimodal data in collaborative maker activities. Proceedings of the 14th Learning Analytics and Knowledge Conference (LAK ’24), 18–22 March 2024, Kyoto, Japan (pp. 922–928). https://doi.org/10.1145/3636555.3636935
Worsley, M., Martinez-Maldonado, R., & D’Angelo, C. (2021). A new era in multimodal learning analytics: Twelve core commitments to ground and grow MMLA. Journal of Learning Analytics, 8(3), 10–27. https://doi.org/10.18608/jla.2021.7361
Yan, L., Martinez-Maldonado, R., Zhao, L., Li, X., & Gašević, D. (2023). Physiological synchrony and arousal as indicators of stress and learning performance in embodied collaborative learning. Proceedings of the 24th International Conference on Artificial Intelligence in Education (AIED 2023), 3–7 July 2023, Tokyo, Japan (pp. 602–614). Springer Cham. https://doi.org/10.1007/978-3-031-36272-9_49
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
