Examining Student Regulation of Collaborative, Computational, Problem-Solving Processes in Open-Ended Learning Environments
DOI:
https://doi.org/10.18608/jla.2021.7230Keywords:
collaborative problem solving, socially shared regulation, natural language processing, process mining, computational modellingAbstract
The integration of computational modelling in science classrooms provides a unique opportunity to promote key 21st century skills including computational thinking (CT) and collaboration. The open-ended, problem-solving nature of the task requires groups to grapple with the combination of two domains (science and computing) as they collaboratively construct computational models. While this approach has produced significant learning gains for students in both science and CT in K–12 settings, the collaborative learning processes students use, including learner regulation, are not well understood. In this paper, we present a systematic analysis framework that combines natural language processing (NLP) of collaborative dialogue, log file analyses of students’ model-building actions, and final model scores. This analysis is used to better understand students’ regulation of collaborative problem solving (CPS) processes over a series of computational modelling tasks of varying complexity. The results suggest that the computational modelling challenges afford opportunities for students to a) explore resource-intensive processes, such as trial and error, to more systematic processes, such as debugging model errors by leveraging data tools, and b) learn from each other using socially shared regulation (SSR) and productive collaboration. The use of such SSR processes correlated positively with their model-building scores. Our paper aims to advance our understanding of collaborative, computational modelling in K–12 science to better inform classroom applications.
References
Amon, M. J., Vrzakova, H., & D’Mello, S. K. (2019). Beyond dyadic coordination: Multimodal behavioral irregularity in groups predicts facets of collaborative problem solving. Cognitive Science, 43(10), e12787. https://doi.org/10.1111/cogs.12787
Awwal, N., Scoular, C., & Alom, M. (2017). An automated system for evaluating 21st century skills using game-based assessments. Proceedings of the 9th International Conference on Education and New Learning Technologies (EDULEARN17), 3–5 July 2017, Barcelona, Spain (pp. 1593–1598). Valencia, Spain: International Academy of Technology, Education and Development (IATED).
Bakhtiar, A., & Hadwin, A. (2020). Dynamic interplay between modes of regulation during motivationally challenging episodes in collaboration. Frontline Learning Research, 8(2), 1–34. https://doi.org/10.14786/flr.v8i2.561
Bakliwal, A., Arora, P., Patil, A., & Varma, V. (2011). Towards enhanced opinion classification using NLP techniques. Proceedings of the Workshop on Sentiment Analysis Where AI Meets Psychology (SAAIP 2011), November 2011, Chiang Mai, Thailand (pp. 101–107). Asian Federation of Natural Language Processing. https://www.aclweb.org/anthology/W11-3715
Bangerter, A. (2004). Using pointing and describing to achieve joint focus of attention in dialogue. Psychological Science, 15(6), 415–419. https://doi.org/10.1111/j.0956-7976.2004.00694.x
Barron, B. (2003). When smart groups fail. The Journal of the Learning Sciences, 12(3), 307–359.
Basu, S., Biswas, G., & Kinnebrew, J. S. (2016). Using multiple representations to simultaneously learn computational thinking and middle school science. Proceedings of the 30th Conference on Artificial Intelligence (AAAI-16), 12–17 February 2016, Phoenix, AZ, USA (pp. 3705–3711). Palo Alto, CA: AAAI Press.
Blackman, N. J.-M., & Koval, J. J. (2000). Interval estimation for Cohen’s kappa as a measure of agreement. Statistics in Medicine, 19(5), 723–741.
Broll, B., Lédeczi, A., Volgyesi, P., Sallai, J., Maroti, M., Carrillo, A., Weeden-Wright, S. L., Vanags, C., Swartz, J. D., & Lu, M. (2017). A visual programming environment for learning distributed programming. Proceedings of the 48th ACM Technical Symposium on Computer Science Education (SIGCSE ’17), 8–11 March 2017, Seattle, WA, USA (pp. 81–86). New York: ACM.
Chiu, M. M., & Khoo, L. (2003). Rudeness and status effects during group problem solving: Do they bias evaluations and reduce the likelihood of correct solutions? Journal of Educational Psychology, 95(3), 506.
Clark, H. H., & Brennan, S. E. (1991). Grounding in communication. In Perspectives on socially shared cognition (pp. 127–149). American Psychological Association. https://doi.org/10.1037/10096-006
De Backer, L., Van Keer, H., & Valcke, M. (2015). Exploring evolutions in reciprocal peer tutoring groups’ socially shared metacognitive regulation and identifying its metacognitive correlates. Learning and Instruction, 38, 63–78.
De Weerdt, J., De Backer, M., Vanthienen, J., & Baesens, B. (2012). A multi-dimensional quality assessment of state-of-the-art process discovery algorithms using real-life event logs. Information Systems, 37(7), 654–676.
Dowell, N., Lin, Y., Godfrey, A., & Brooks, C. (2020). Exploring the relationship between emergent sociocognitive roles, collaborative problem-solving skills and outcomes: A group communication analysis. Journal of Learning Analytics, 7(1), 38–57. https://doi.org/10.18608/jla.2020.71.4
Dowell, N. M., Nixon, T. M., & Graesser, A. C. (2019). Group communication analysis: A computational linguistics approach for detecting sociocognitive roles in multiparty interactions. Behavior Research Methods, 51(3), 1007–1041.
Emara, M., Tscholl, M., Dong, Y., & Biswas, G. (2017). Analyzing students’ collaborative regulation behaviors in a classroom-integrated open ended learning environment. In B. K. Smith, M. Borge, E. Mercier, & K. Y. Lim (Eds.), Making a Difference: Prioritizing Equity and Access in CSCL: Proceedings of the 12th International Conference on Computer Supported Collaborative Learning (CSCL 2017) 18–22 June 2017, Philadelphia, PA, USA (pp. 319–326). International Society of the Learning Sciences.
Emara, M., Grover, S., Hutchins, N. M., Biswas, G., & Snyder, C. (2020). Examining students’ debugging and regulation processes during collaborative computational modeling in science. 14th International Conference of the Learning Sciences (ICLS ’20), 19–23 June 2020, Nashville, TN, USA. International Society of the Learning Sciences.
Ferreira, R., Kovanović, V., Gašević, D., & Rolim, V. (2018). Towards combined network and text analytics of student discourse in online discussions. In C. Penstein Rosé et al. (Eds.), Artificial intelligence in education (Proceedings of AIED 2018). Lecture Notes in Computer Science, vol. 10947. Springer, Cham. https://doi.org/10.1007/978-3-319-93843-1_9
Fischer, C., Pardos, Z. A., Baker, R. S., Williams, J. J., Smyth, P., Yu, R., Slater, S., Baker, R., & Warschauer, M. (2020). Mining big data in education: Affordances and challenges. Review of Research in Education, 44(1), 130–160. https://doi.org/10.3102/0091732X20903304
Gašević, D., Jovanović, J., Pardo, A., & Dawson, S. (2017). Detecting learning strategies with analytics: Links with self-reported measures and academic performance. Journal of Learning Analytics, 4(2), 113–128. https://doi.org/10.18608/jla.2017.42.10
Gobert, J., Slotta, J., Clarke, J., Dede, C., Gijlers, H., Saab, N., Van Joolingen, W., De Jong, T., & Koedinger, K. (2007). Fostering peer collaboration with technology. In C. Chinn, G. Erkens, & S. Puntambekar (Eds.), Proceedings of the 7th International Conference on Computer-Supported Collaborative Learning (CSCL 2007), 16–21 July 2007, New Brunswick, NJ, USA (pp. 23–27). International Society of the Learning Sciences.
Greene, J. A., & Azevedo, R. (2009). A macro-level analysis of SRL processes and their relations to the acquisition of a sophisticated mental model of a complex system. Contemporary Educational Psychology, 34(1), 18–29.
Grover, S., Bienkowski, M., Tamrakar, A., Siddiquie, B., Salter, D., & Divakaran, A. (2016, April). Multimodal analytics to study collaborative problem solving in pair programming. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼ16), 25–29 April 2016, Edinburgh, UK (pp. 516–517). New York: ACM.
Grover, S., Basu, S., Bienkowski, M., Eagle, M., Diana, N., & Stamper, J. (2017). A framework for using hypothesis-driven approaches to support data-driven learning analytics in measuring computational thinking in block-based programming environments. ACM Transactions on Computing Education (TOCE), 17(3), 1–25.
Grover, S., Hutchins, N., Biswas, G., Snyder, C., & Emara, M. (2019). Examining synergistic learning of physics and computational thinking through collaborative problem solving in computational modeling. Proceedings of the American Educational Research Association Annual Conference (AERA 2019), 5–9 April 2019, Toronto, ON, Canada. https://www.researchgate.net/publication/332427176_Examining_Synergistic_Learning_of_Physics_and_Computational_Thinking_through_Collaborative_Problem_Solving_in_Computational_Modeling/stats
Grover, S., & Pea, R. (2013). Computational thinking in K–12: A review of the state of the field. Educational Researcher, 42(1), 38–43.
Günther, C. W., & Van Der Aalst, W. M. (2007). Fuzzy mining: Adaptive process simplification based on multi-perspective metrics. In G. Alonso, P. Dadam, & M. Rosemann (Eds.), Proceedings of the 5th International Conference on Business Process Management (BPM 2007), 24–28 September 2007, Brisbane, Australia (pp. 328–343). Springer.
Hadwin, A., Järvelä, S., & Miller, M. (2018). Self-regulation, co-regulation, and shared regulation in collaborative learning environments. In D. H. Schunk & J. A. Greene (Eds.), Educational psychology handbook series. Handbook of self-regulation of learning and performance (pp. 83–106). Routledge/Taylor & Francis Group.
Hambrusch, S., Hoffmann, C., & Korb, J. T. (2009). A multidisciplinary approach towards computational thinking for science majors. Proceedings of the 40th ACM Technical Symposium on Computer Science Education (SIGCSE 09), 4–7 March 2009, Chattanooga, TN, USA (pp. 183–187). New York: ACM. https://doi.org/10.1145/1508865.1508931
Hao, J., Chen, L., Flor, M., Liu, L., & von Davier, A. A. (2017). CPS‐Rater: Automated sequential annotation for conversations in collaborative problem‐solving activities. ETS Research Report Series, 2017(1). https://doi.org/1-9. 10.1002/ets2.12184
Henderson, P. B., Cortina, T. J., & Wing, J. M. (2007). Computational thinking. Proceedings of the 38th ACM Technical Symposium on Computer Science Education (SIGCSE 07), 7–10 March 2007, Covington, KY, USA (pp. 195–196). New York: ACM.
Hmelo-Silver, C. E., Chernobilsky, E., & Jordan, R. (2008). Understanding collaborative learning processes in new learning environments. Instructional Science, 36(5–6), 409–430.
Hutchins, N., Biswas, G., Grover, S., Basu, S., & Snyder, C. (2019). A systematic approach for analyzing students’ computational modeling processes in C2STEM. In S. Isotani, E. Millán, A. Ogan, P. Hastings, B. McLaren, & R. Luckin (Eds.), Proceedings of the 20th International Conference on Artificial Intelligence in Education (AIED 2019), 25–29 June 2019, Chicago, IL, USA (pp. 116–121). Springer. https://doi.org/10.1007/978-3-030-23207-8_22
Hutchins, N. M., Biswas, G., Zhang, N., Snyder, C., Lédeczi, Á., & Maróti, M. (2020a). Domain-specific modeling languages in computer-based learning environments: A systematic approach to support science learning through computational modeling. International Journal of Artificial Intelligence in Education, 30, 537–580. https://doi.org/10.1007/s40593-020-00209-z
Hutchins, N. M., Biswas, G., Maróti, M., Lédeczi, Á., Grover, S., Wolf, R., Blair, K. P., Chin, D., Conlin, L., Basu, S., & McElhaney, K. (2020b). C2STEM: A system for synergistic learning of physics and computational thinking. Journal of Science Education and Technology, 29(1), 83–100. https://doi.org/10.1007/s10956-019-09804-9
Iiskala, T., Vauras, M., Lehtinen, E., & Salonen, P. (2011). Socially shared metacognition of dyads of pupils in collaborative mathematical problem-solving processes. Learning and Instruction, 21(3), 379–393. https://doi.org/10.1016/j.learninstruc.2010.05.002
Isohätälä, J., Järvenoja, H., & Järvelä, S. (2017). Socially shared regulation of learning and participation in social interaction in collaborative learning. International Journal of Educational Research, 81, 11–24. https://doi.org/10.1016/j.ijer.2016.10.006
Järvelä, S., Malmberg, J., & Koivuniemi, M. (2016). Recognizing socially shared regulation by using the temporal sequences of online chat and logs in CSCL. Learning and Instruction, 42, 1–11.
Jona, K., Wilensky, U., Trouille, L., Horn, M. S., Orton, K., Weintrop, D., & Beheshti, E. (2014). Embedding computational thinking in science, technology, engineering, and math (CT-STEM). Future Directions in Computer Science Education Summit Meeting, Orlando, FL.
Kapur, M. (2016). Examining productive failure, productive success, unproductive failure, and unproductive success in learning. Educational Psychologist, 51(2), 289–299. https://doi.org/10.1080/00461520.2016.1155457
Kinnebrew, J., Segedy, J. R., & Biswas, G. (2017). Integrating model-driven and data-driven techniques for analyzing learning behaviors in open-ended learning environments. IEEE Transactions on Learning Technologies, 10(2), 140–153. https://doi.org/10.1109/TLT.2015.2513387
Klahr, D., & Carver, S. M. (1988). Cognitive objectives in a LOGO debugging curriculum: Instruction, learning, and transfer. Cognitive Psychology, 20(3), 362–404.
Kneser, C., Pilkington, R., & Treasure-Jones, T. (2001). The tutor’s role: An investigation of the power of exchange structure analysis to identify different roles in CMC seminars. International Journal of Artificial Intelligence in Education, 12(63–84).
Kurniati, A. P., Kusuma, G., & Wisudawan, G. (2016). Implementing heuristic miner for different types of event logs. International Journal of Applied Engineering Research, 11, 5523–5529. https://www.researchgate.net/publication/301998930_Implementing_Heuristic_Miner_for_Different_Types_of_Event_Logs
Landau, R. (2006). Computational physics: A better model for physics education? Computing in Science & Engineering, 8(5), 22–30.
Lin, K.-Y., Yu, K.-C., Hsiao, H.-S., Chu, Y.-H., Chang, Y.-S., & Chien, Y.-H. (2015). Design of an assessment system for collaborative problem solving in STEM education. Journal of Computers in Education, 2(3), 301–322. https://doi.org/10.1007/s40692-015-0038-x
Liu, Z., Zhi, R., Hicks, A., & Barnes, T. (2017). Understanding problem solving behavior of 6–8 graders in a debugging game. Computer Science Education, 27(1), 1–29. https://doi.org/10.1080/08993408.2017.1308651
Lobczowski, N. G., Allen, E. M., Firetto, C. M., Greene, J. A., & Murphy, P. K. (2020). An exploration of social regulation of learning during scientific argumentation discourse. Contemporary Educational Psychology, 101925. https://doi.org/10.1016/j.cedpsych.2020.101925
Loksa, D., & Ko, A. J. (2016). The role of self-regulation in programming problem solving process and success. Proceedings of the 12th Annual Conference on International Computing Education Research (ICER 2016), 8–12 September 2016, Melbourne, Australia (pp. 83–91). New York: ACM. https://doi.org/10.1145/2960310.2960334
Malmberg, J., Järvelä, S., & Järvenoja, H. (2017). Capturing temporal and sequential patterns of self-, co-, and socially shared regulation in the context of collaborative learning. Contemporary Educational Psychology, 49, 160–174.
Malmberg, J., Järvelä, S., & Kirschner, P. A. (2014). Elementary school students’ strategic learning: Does task-type matter? Metacognition and Learning, 9(2), 113–136.
McNamara, D. S. (2011). Computational methods to extract meaning from text and advance theories of human cognition. Topics in Cognitive Science, 3(1), 3–17. https://doi.org/10.1111/j.1756-8765.2010.01117.x
Mohammed, M., & Omar, N. (2020). Question classification based on Bloom’s taxonomy cognitive domain using modified TF-IDF and word2vec. PloS One, 15(3), e0230442. https://doi.org/10.1371/journal.pone.0230442
Molenaar, I., Chiu, M. M., Sleegers, P., & van Boxtel, C. (2011). Scaffolding of small groups’ metacognitive activities with an avatar. International Journal of Computer-Supported Collaborative Learning, 6(4), 601–624. https://doi.org/10.1007/s11412-011-9130-z
Molenaar, I., & Chiu, M. M. (2014). Dissecting sequences of regulation and cognition: Statistical discourse analysis of primary school children’s collaborative learning. Metacognition and Learning, 9(2), 137–160. https://doi.org/10.1007/s11409-013-9105-8
Noroozi, O., Alikhani, I., Järvelä, S., Kirschner, P. A., Juuso, I., & Seppänen, T. (2019). Multimodal data to design visual learning analytics for understanding regulation of learning. Computers in Human Behavior, 100, 298–304.
Oshima, J., Tsunakawa, T., & Oshima, R. (2020). An assessment of idea emergence in subject-matter collaborative learning. Frontiers in Education, 5, 21.
Paans, C., Onan, E., Molenaar, I., Verhoeven, L., & Segers, E. (2019). How social challenges affect children’s regulation and assignment quality in hypermedia: A process mining study. Metacognition and Learning, 14(2), 189–213.
Papert, S., & Harel, I. (1991). Situating constructionism. Constructionism, 36(2), 1–11.
Peters-Burton, E. E., Cleary, T. J., & Kitsantas, A. (2018). Computational thinking in the context of science and engineering practices: A self-regulated learning approach. Digital Technologies: Sustainable Innovations for Improving Teaching and Learning (pp. 223–240). Springer.
Pino-Pasternak, D., Whitebread, D., & Neale, D. (2018). The role of regulatory, social, and dialogic dynamics on young children’s productive collaboration in group problem solving. New Directions for Child and Adolescent Development, 2018(162), 41–66.
Praharaj, S., Scheffel, M., Drachsler, H., & Specht, M. (2018). Multimodal analytics for real-time feedback in co-located collaboration. Proceedings of the 13th European Conference on Technology Enhanced Learning (EC-TEL 2018), 3–5 September 2018, Leeds, UK (pp. 187–201). Lecture Notes in Computer Science, vol. 11082, Springer.
Redish, E. F., & Wilson, J. M. (1993). Student programming in the introductory physics course: M.U.P.P.E.T. American Journal of Physics, 61(3), 222–232. https://doi.org/10.1119/1.17295
Repenning, A., Webb, D., & Ioannidou, A. (2010). Scalable game design and the development of a checklist for getting computational thinking into public schools. Proceedings of the 41st ACM Technical Symposium on Computer Science Education (SIGCSE 10), 10–13 March 2010, Milwaukee, WI, USA (pp. 265–269). New York: ACM. https://doi.org/10.1145/1734263.1734357
Roschelle, J., & Teasley, S. D. (1995). The construction of shared knowledge in collaborative problem solving. In J. L. Schnase & E. L. Cunnius (Eds.), Proceedings of the 1st International Conference on Computer-Support for Collaborative Learning (CSCL ’95), 17–20 October 1995, Bloomington, IN, USA (pp. 69–97). International Society of the Learning Sciences. Mahwah, NJ: Lawrence Erlbaum Associates.
Rosé, C., Wang, Y.-C., Cui, Y., Arguello, J., Stegmann, K., Weinberger, A., & Fischer, F. (2008). Analyzing collaborative learning processes automatically: Exploiting the advances of computational linguistics in computer-supported collaborative learning. International Journal of Computer-Supported Collaborative Learning, 3(3), 237–271.
Schneider, B., Sharma, K., Cuendet, S., Zufferey, G., Dillenbourg, P., & Pea, R. (2018). Leveraging mobile eye-trackers to capture joint visual attention in co-located collaborative learning groups. International Journal of Computer-Supported Collaborative Learning, 13(3), 241–261. https://doi.org/10.1007/s11412-018-9281-2
Schoor, C., & Bannert, M. (2012). Exploring regulatory processes during a computer-supported collaborative learning task using process mining. Computers in Human Behavior, 28(4), 1321–1331.
Segedy, J. R., Kinnebrew, J. S., & Biswas, G. (2015). Using coherence analysis to characterize self-regulated learning behaviours in open-ended learning environments. Journal of Learning Analytics, 2(1), 13–48. https://doi.org/10.18608/jla.2015.21.3
Sengupta, P., Kinnebrew, J. S., Basu, S., Biswas, G., & Clark, D. (2013). Integrating computational thinking with K–12 science education using agent-based computation: A theoretical framework. Education and Information Technologies, 18(2), 351–380.
Siadaty, M., Gašević, D., & Hatala, M. (2016). Associations between technological scaffolding and micro-level processes of self-regulated learning: A workplace study. Computers in Human Behavior, 55, 1007–1019.
Snyder, C., Hutchins, N., Biswas, G., Emara, M., Grover, S., & Conlin, L. (2019). Analyzing students’ synergistic learning processes in physics and CT by collaborative discourse analysis. In K. Lund, G. Niccolai, E. Lavoué, C. Hmelo-Silver, G. Gweon, & M. Baker (Eds.), A Wide Lens: Combining Embodied, Enactive, Extended, and Embedded Learning in Collaborative Settings. Proceedings of the 13th International Conference on Computer Supported Collaborative Learning (CSCL 2019), 17–21 June 2019, Lyon, France (pp. 360–367). International Society of the Learning Sciences. https://doi.dx.org/10.22318/cscl2019.360
Sobocinski, M., Järvelä, S., Malmberg, J., Dindar, M., Isosalo, A., & Noponen, K. (2020). How does monitoring set the stage for adaptive regulation or maladaptive behavior in collaborative learning? Metacognition and Learning, 15, 99–127. https://doi.org/10.1007/s11409-020-09224-w
Sobocinski, M., Malmberg, J., & Järvelä, S. (2017). Exploring temporal sequences of regulatory phases and associated interactions in low-and high-challenge collaborative learning sessions. Metacognition and Learning, 12(2), 275–294.
Soderstrom, N. C., & Bjork, R. A. (2015). Learning versus performance: An integrative review. Perspectives on Psychological Science, 10(2), 176–199. https://doi.org/10.1177/1745691615569000
Sonnenberg, C., & Bannert, M. (2015). Discovering the effects of metacognitive prompts on the sequential structure of SRL-processes using process mining techniques. Journal of Learning Analytics, 2(1), 72–100. https://doi.org/10.18608/jla.2015.21.5
Stewart, A. E., Vrzakova, H., Sun, C., Yonehiro, J., Stone, C. A., Duran, N. D., . . ., & D’Mello, S. K. (2019). I say, you say, we say: Using spoken language to model socio-cognitive processes during computer-supported collaborative problem solving. Proceedings of the ACM on Human–Computer Interaction (Volume 3, Issue CSCW, Article No. 194, pp. 1–19), November 2019. New York: ACM. https://doi.org/10.1145/3359296
Sullivan, F. R., & Keith, P. K. (2019). Exploring the potential of natural language processing to support microgenetic analysis of collaborative learning discussions. British Journal of Educational Technology, 50(6), 3047–3063.
Sun, C., Shute, V. J., Stewart, A., Yonehiro, J., Duran, N., & D’Mello, S. (2020). Towards a generalized competency model of collaborative problem solving. Computers, & Education, 143, 103672. https://doi.org/10.1016/j.compedu.2019.103672
Toulmin, S. E. (2001). Return to reason. Cambridge, MA: Harvard University Press.
Van den Bossche, P., Gijselaers, W., Segers, M., Woltjer, G., & Kirschner, P. (2011). Team learning: Building shared mental models. Instructional Science, 39(3), 283–301. https://doi.org/10.1007/s11251-010-9128-3
Van der Aalst, W. M. (2011). Process discovery: An introduction. In Process mining: Discovery, conformance and enhancement of business processes (pp. 125–156). Springer. https://doi.org/10.1007/978-3-642-19345-3_5
Verheij, B. (2005). Evaluating arguments based on Toulmin’s scheme. Argumentation, 19(3), 347-371. https://doi.org/10.1007/s10503-005-4421-z
von Davier, A. A., Hao, J., Liu, L., & Kyllonen, P. (2017). Interdisciplinary research agenda in support of assessment of collaborative problem solving: Lessons learned from developing a collaborative science assessment prototype. Computers in Human Behavior, 76, 631–640. https://doi.org/10.1016/j.chb.2017.04.059
Weijters, A. J. M. M., van Der Aalst, W. M., & De Medeiros, A. A. (2006). Process mining with the heuristics miner-algorithm. Technische Universiteit Eindhoven, Tech. Rep. WP, 166, 1–34.
Weinberger, A., & Fischer, F. (2006). A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. Computers and Education, 46(1), 71–95. https://doi.org/10.1016/j.compedu.2005.04.003
Weintrop, D., Beheshti, E., Horn, M., Orton, K., Jona, K., Trouille, L., & Wilensky, U. (2016). Defining computational thinking for mathematics and science classrooms. Journal of Science Education and Technology, 25(1), 127–147. https://doi.org/10.1007/s10956-015-9581-5
Werner, L., McDowell, C., & Denner, J. (2013). A first step in learning analytics: Pre-processing low-level Alice logging data of middle school students. Journal of Educational Data Mining, 5(2), 11–37. https://doi.org/10.5281/zenodo.3554631
Williams, L., Wiebe, E., Yang, K., Ferzli, M., & Miller, C. (2002). In support of pair programming in the introductory computer science course. Computer Science Education, 12(3), 197–212.
Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. Metacognition in Educational Theory and Practice, 93, 27–30.
Winters, F. I., & Alexander, P. A. (2011). Peer collaboration: The relation of regulatory behaviors to learning with hypermedia. Instructional Science, 39(4), 407–427. https://doi.org/10.1007/s11251-010-9134-5
Wise, A. F., Azevedo, R., Stegmann, K., Malmberg, J., Rosé, C. P., Mudrick, N., Taub, M., Martin, S. A., Farnsworth, J., & Mu, J. (2015). CSCL and learning analytics: Opportunities to support social interaction, self-regulation and socially shared regulation. International Society of the Learning Sciences.
Worsley, M., & Blikstein, P. (2011). What’s an expert? Using learning analytics to identify emergent markers of expertise through automated speech, sentiment and sketch analysis. In M. Pechenizkiy et al. (Eds.), Proceedings of the 4th Annual Conference on Educational Data Mining (EDM2011), 6–8 July 2011, Eindhoven, Netherlands (pp. 235–240). International Educational Data Mining Society.
Xie, K., Di Tosto, G., Lu, L., & Cho, Y. S. (2018). Detecting leadership in peer-moderated online collaborative learning through text mining and social network analysis. The Internet and Higher Education, 38, 9–17.
Yett, B., Hutchins, N., Stein, G., Zare, H., Snyder, C., Biswas, G., Metelko, M., & Lédeczi, Á. (2020). A hands-on cybersecurity curriculum using a robotics platform. Proceedings of the 51st ACM Technical Symposium on Computer Science Education (SIGCSE ’20), 11–14 March 2020, Portland, OR, USA (pp. 1040–1046). New York: ACM.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2021 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
