Investigating Collaboration as a Process with Theory-driven Learning Analytics
DOI:
https://doi.org/10.18608/jla.2020.71.5Keywords:
collaborative learning assessment, the collaborative cognitive load theory, collective learning, social network analysisAbstract
Although collaboration is considered a key 21st-century skill, oftentimes it is only assessed through the outcome measures of individual learners. In this paper, we draw upon collaborative cognitive load theory (CCLT) to explain the process of collaboration in learning from the point of view of the collective, rather than an individual learner. Using CCLT we suggest a new method of measuring the process of collaboration regardless of its outcome measures. Our approach — Collaborative Learning as a Process (CLaP) — uses social network analysis to evaluate the balance between interactivity gains and coordination costs of learner communities. Here, we demonstrate the approach using real-world data derived from the digital tracks of two online discussion communities. We argue that our conceptual approach can enable instructors and learners to unlock the black box of collaboration in learning. It has the potential to support the development of learner skills that go beyond cognition. We conclude the paper with the results of our investigation of the value of the approach to the online module instructor.
References
Ausubel, D. P. (1968). Educational psychology: A cognitive view. New York: Holt, Rinehart & Winston.
Bernstein, E., Shore, J., & Lazer, D. (2018). How intermittent breaks in interaction improve collective intelligence, PNAS: Proceedings of the National Academy of Sciences of the United States of America, 115(35), 8734–8739. https://dx.doi.org/10.1073/pnas.1802407115
Borgatti, S. P., Mehra, A., Brass, D. J., & Labianca, G. (2009). Network analysis in the social sciences. Science, 323(5916), 892–895. http://dx.doi.org/10.1126/science.1165821
Casner-Lotto, J., & Barrington, L. (2006). Are they really ready to work? Employers’ perspectives on the basic knowledge and applied skills of new entrants to the 21st century US workforce. Washington, DC: Partnership for 21st Century Skills.
Chiocchio, F., & Essiembre, H. (2009). Cohesion and performance: A meta-analytic review of disparities between project teams, production teams, and service teams. Small Group Research, 40(4), 382–420. http://dx.doi.org/10.1177/1046496409335103
Chounta, I. A., Hecking, T., Hoppe, H. U., & Avouris, N. (2014). Two make a network: Using graphs to assess the quality of collaboration of dyads. CYTED-RITOS International Workshop on Groupware, 53–66. Cham, Switzerland: Springer. https://dx.doi.org/10.1007/978-3-319-10166-8_5
Claros, I., Cobos, R., & Collazos, C. A. (2016). An approach based on social network analysis applied to a collaborative learning experience. IEEE Transactions on Learning Technologies, 9(2), 190–195. http://dx.doi.org/10.1109/TLT.2015.2453979
Collazos, C., Padilla, N., Pozzi, F., Guerrero, L., & Gutierrez, F. (2014). Design guidelines to foster cooperation in digital environments. Technology, Pedagogy and Education, 23(3), 375–396. https://dx.doi.org/10.1080/1475939X.2014.943277
Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24, 87–114. http://dx.do.org/10.1017/S0140525X01003922
Cukurova, M. (2018). A syllogism for designing collaborative learning technologies in the age of AI and multimodal data. Proceedings of the 13th European Conference on Technology Enhanced Learning (EC-TEL 2018), 3–5 September 2018, Leeds, UK (pp. 291–296). Lecture Notes in Computer Science, Springer. https://dx.doi.org/10.1007/978-3-319-98572-5_22
Cukurova, M., Bennett, J. M., & Abrahams, I. Z. (2018). Students’ knowledge acquisition and ability to apply knowledge into different science contexts in two different independent learning settings. Research in Science and Technological Education, 36(1), 17–34. http://dx.doi.org/0.1080/02635143.2017.1336709
Cukurova, M., Luckin, R., Millán, E., & Mavrikis, M. (2018). The NISPI framework: Analysing collaborative problem-solving from students’ physical interactions. Computers & Education, 116, 93–109. https://dx.doi.org/10.1016/j.compedu.2017.08.007
Cukurova, M., Luckin, R., Millán, E., Mavrikis, M., & Spikol, D. (2017). Diagnosing collaboration in practice-based learning: Equality and intra-individual variability of physical interactivity. In É. Lavoué, H. Drachsler, K. Verbert, J. Broisin, & M. Pérez-Sanagustín (Eds.), Data Driven Approaches in Digital Education, pp. 30–42. EC-TEL 2017. Lecture Notes in Computer Science, vol. 10474. Springer, Cham. https://dx.doi.org/10.1007/978-3-319-66610-5_3
Damon, W., & Phelps, E. (1989). Critical distinctions among three approaches to peer education. International Journal of Educational Research, 13(1), 9–19. https://dx.doi.org/10.1016/0883-0355(89)90013-X
Dede, C. (2010). Comparing frameworks for 21st century skills. 21st Century Skills: Rethinking How Students Learn, 20, 51–76.
De Laat, M., Lally, V., Lipponen, L., & Simons, R. J. (2007). Investigating patterns of interaction in networked learning and computer-supported collaborative learning: A role for social network analysis. International Journal of Computational Supported Collaborative Learning, 2(1), 87–103. https://dx.doi.org/10.1007/s11412-007-9006-4
Dillenbourg, P. (1999). What do you mean by “collaborative learning”? In P. Dillenbourg (Ed.), Collaborative learning: Cognitive and computational approaches (pp. 1–16). Amsterdam, Netherlands: Elsevier.
Dunbar, R. I. M. (1992). Neocortex size as a constraint on group size in primates. Journal of Human Evolution, 22(6), 469–493. https://dx.doi.org/10.1016/0047-2484(92)90081-J
Easley, D., & Kleinberg, J. (2010). Networks, crowds, and markets: Reasoning about a highly connected world. Cambridge, UK: Cambridge University Press.
Farrington, C. A., Roderick, M., Allensworth, E., Nagaoka, J., Keyes, T. S., Johnson, D. W., & Beechum, N. O. (2012). Teaching adolescents to become learners. The role of noncognitive factors in shaping school performance: A critical literature review. Chicago, IL: University of Chicago Consortium on Chicago School Research.
Freeman, L. C. (1978). Centrality in social networks conceptual clarification. Social Networks, 1(3), 215–239.
Gardner, J. (2011). Assessment and learning. Thousand Oaks, CA: Sage Publications.
Goyal, S., & Vega-Redondo, F. (2005). Network formation and social coordination. Games and Economic Behavior, 50(2), 178–207.
Hutchins, E. (1996). Cognition in the wild. Cambridge, MA: MIT Press.
Ickes, W. (2002). Social cognition: Subjective and intersubjective paradigms. New Review of Social Psychology, 1(June), 112–121.
Ickes, W., & Gonzalez, R. (1996). “Social” cognition and social cognition: From the subjective to the intersubjective. In J. L. Nye & A. M. Brower (Eds.), What’s social about social cognition? Research on socially shared cognition in small groups (pp. 285–308). Thousand Oaks, CA: Sage Publications. https://dx.doi.org/10.4135/9781483327648.n12
Janssen, J., Kirschner, F., Erkens, G., Kirschner, P. A., & Paas, F. (2010). Making the black box of collaborative learning transparent: Combining process-oriented and cognitive load approaches. Educational Psychology Review, 22(2), 139–154. https://dx.doi.org/10.1007/s10648-010-9131-x
Johnson, D. W., & Johnson, R. T. (2002). Learning together and alone: Overview and meta-analysis. Asia Pacific Journal of Education, 22, 95–105. https://dx.doi.org/10.1080/0218879020220110
Johnson, D. W., Johnson, R. T., Roseth, C. J., & Seob Shin, T. (2014). The relationship between motivation and achievement in interdependent situations. Journal of Applied Social Psychology, 44(9), 622–633. https://dx.doi.org/10.1111/jasp.12280
Kalman, R. E. (1963). Mathematical description of linear dynamical systems. Journal of the Society for Industrial and Applied Mathematics, Series A: Control, 1(2), 152–192. https://dx.doi.org/10.1016/S0076-5392(08)63251-8
Kent, C., Laslo, E., & Rafaeli, S. (2016). Interactivity in online discussions and learning outcomes. Computers & Education, 97, 116–128. https://dx.doi.org/10.1016/j.compedu.2016.03.002
Kent, C., & Rechavi, A. (2018). Deconstructing online social learning: Network analysis of the creation, consumption and organization types of interaction. International Journal of Research & Method in Education, 43(1), 1–22. http://dx.doi.org/10.1080/1743727X.2018.1524867
Kent, C., Rechavi, A., & Rafaeli, R. (2019). Networked learning analytics: A theoretically informed methodology for collaborative learning analytics. In Y. Kali, A. Baram-Tsabari, & A., Schejter (Eds.), Learning in a networked society: Spontaneous and designed technology enhanced learning communities (pp. 145–175). Springer. https://dx.doi.org/10.1007/978-3-030-14610-8_9
Kester, L., & Paas, F. (2005). Instructional interventions to enhance collaboration in powerful learning environments. Computers in Human Behavior, 21, 689–696. https://dx.doi.org/10.1016/j.chb.2004.11.008
Kirschner, P. A., Kirschner, F., & Janssen, J. (2014). The collaboration principle in multimedia learning. In R. Mayer (Ed.), The Cambridge handbook of multimedia learning, 2nd ed. (pp. 547–575). New York: Cambridge University Press. https://dx.doi.org/10.1017/CBO9781139547369.027
Kirschner, F., Paas, F., & Kirschner, P. A. (2011). Task complexity as a driver for collaborative learning efficiency: The collective working‐memory effect. Applied Cognitive Psychology, 25(4), 615–624. https://dx.doi.org/10.1002/acp.1730
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75–86. https://dx.doi.org/10.1207/s15326985ep4102_1
Kirschner, P. A., Sweller, J., Kirschner, F., & Zambrano, R. J. (2018). From cognitive load theory to collaborative cognitive load theory. International Journal of Computer-Supported Collaborative Learning, 13, 213–233. https://dx.doi.org/10.1007/s11412-018-9277-y
Knight, S., Buckingham Shum, S., & Littleton, K. (2013). Epistemology, pedagogy, assessment and learning analytics, Proceedings of the 3rd International Conference on Learning Analytics and Knowledge (LAK ’13), 8–12 April 2013, Leuven, Belgium (pp. 75–84). New York: ACM. https://dx.doi.org/10.1145/2460296.2460312
Kwon, K., Liu, Y. H., & Johnson, L. P. (2014). Group regulation and social-emotional interactions observed in computer supported collaborative learning: Comparison between good vs. poor collaborators. Computers & Education, 78, 185–200. https://dx.doi.org/10.1016/j.compedu.2014.06.004
Liu, Y. Y., Slotine, J. J., & Barabási, A. L. (2011). Controllability of complex networks. Nature, 473, 167–173. https://dx.doi.org/10.1038/nature10011
Martinez, A., Dimitriadis, Y., Rubia, B., Gomez, E., & De La Fuente, P. (2003). Combining qualitative evaluation and social network analysis for the study of classroom social interactions. Computers & Education, 41(4), 353–368. https://dx.doi.org/10.1016/j.compedu.2003.06.001
Murnane, R. J., & Levy, F. (1996). Teaching the new basic skills: Principles for educating children to thrive in a changing economy. New York: Free Press.
Newman, M. E. J. (2001). The structure of scientific collaboration networks. PNAS: Proceedings of the National Academy of Sciences of the United States of America, 98(2), 404–409. https://dx.doi.org/10.1073/pnas.021544898
OECD (Organisation for Economic Co-operation and Development). (2015). PISA 2015 collaborative problem solving framework. Oxford, UK: OECD Publishing.
Paas, F., & Sweller, J. (2012). An evolutionary upgrade of cognitive load theory: Using the human motor system and collaboration to support the learning of complex cognitive tasks. Educational Psychology Review, 24(1), 27–45. https://dx.doi.org/10.1007/s10648-011-9179-2
Palincsar, A. S. (1998). Social constructivist perspectives on teaching and learning. Annual Review of Psychology, 49(1), 345–375. https://dx.doi.org/10.1146/annurev.psych.49.1.345
Peterson, L. R., & Peterson, M. J. (1959). Short-term retention of individual verbal items. Journal of Experimental Psychology, 58, 193–198. http://dx.doi.org/10.1037/h0049234
Rabbany, R., Elatia, S., Takaffoli, M., & Zaïane, O. R. (2013). Collaborative learning of students in online discussion forums: A social network analysis perspective. In A. Peña-Ayala (Ed.), Educational data mining: Applications and trends (pp. 441–466). Springer. https://dx.doi.org/1007/978-3-319-02738-8_16
Roschelle, J., & Teasley, S. D. (1995). The construction of shared knowledge in collaborative problem solving. In C. E. O’Malley (Ed.), Computer-supported collaborative learning (pp. 69–197). Berlin: Springer-Verlag. https://dx.doi.org/10.1007/978-3-642-85098-1_5
Siemens, G. (2005). A learning theory for the digital age. Instructional Technology and Distance Education, 2(1), 3–10.
Slavin, R. E. (2014). Cooperative learning and academic achievement: Why does groupwork work? Anales de Psicología/Annals of Psychology, 30, 785–791. https://dx.doi.org/10.6018/analesps.30.3.201201
Soboroff, S. D. (2012). Group size and the trust, cohesion, and commitment of group members [Unpublished doctoral dissertation]. University of Iowa.
Stahl, G. (2010). Group cognition as a foundation for the new science of learning. In M. S. Khine & I. M. Saleh (Eds.), New science of learning: Cognition, computers and collaboration in education (pp. 23–44). https://dx.doi.org/10.1007/978-1-4419-5716-0
Strijbos, J. W. (2011). Assessment of (computer-supported) collaborative learning. IEEE Transactions on Learning Technologies, 4(1), 59–73. https://dx.doi.org/10.1007/s40692-014-0009-7
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. https://dx.doi.org/10.1207/s15516709cog1202_4
Tsai, W. (2002). Social structure of “coopetition” within a multiunit organization: Coordination, competition, and intraorganizational knowledge sharing. Organization Science, 13(2), 179–190. https://dx.doi.org/10.1287/orsc.13.2.179.536
Tulin, M., Pollet, T. V., & Lehmann-Willenbrock, N. (2018). Perceived group cohesion versus actual social structure: A study using social network analysis of egocentric Facebook networks. Social Science Research, 75, 161–175. https://dx.doi.org/10.1016/j.ssresearch.2018.04.004
Veerman, A., & Veldhuis-Diermanse, E. (2001). Collaborative learning through computer-mediated communication in academic education. In Proceedings of the 1st European Conference on Computer-Supported Collaborative Learning (Euro-CSCL 2001), 22–24 March 2001, Maastricht, Netherlands (pp. 625–632). International Society of the Learning Sciences.
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological process. Cambridge, MA: Harvard University Press.
Wasserman, S., & Faust, K. (1994). Social network analysis: Methods and applications. Cambridge, UK: Cambridge University Press.
West, M. R., Kraft, M. A., Finn, A. S., Martin, R. E., Duckworth, A. L., Gabrieli, C. F., & Gabrieli, J. D. (2016). Promise and paradox: Measuring students’ non-cognitive skills and the impact of schooling. Educational Evaluation and Policy Analysis, 38(1), 148–170. https://dx.doi.org/10.3102/0162373715597298
Woolley, A. W., Chabris, C. F., Pentland, A., Hashmi, N., & Malone, T. W. (2010). Evidence for a collective intelligence factor in the performance of human groups. Science, 330(6004), 686–688. http://dx.doi./org/10.1126/science.1193147
Downloads
Published
How to Cite
Issue
Section
License
TEST
