Exploring the Relationship between Emergent Sociocognitive Roles, Collaborative Problem- Solving Skills and Outcomes: A Group Communication Analysis
DOI:
https://doi.org/10.18608/jla.2020.71.4Keywords:
Group communication analysis, sociocognitive roles, computational linguistics, collaborative problem-solving, STEMAbstract
Collaborative problem-solving (CPS) has become an essential component of today’s knowledge-based, innovation- centred economy and society. As such, communication and CPS are now considered critical 21st century skills and incorporated into educational practice, policy, and research. Despite general agreement that these are important skills, there is less agreement on how to capture sociocognitive processes automatically during team interactions to gain a better understanding of their relationship with CPS outcomes. The availability of naturally occurring educational discourse data within online CPS platforms presents a golden opportunity to advance understanding about online learner sociocognitive roles and ecologies. In this paper, we explore the relationship between emergent sociocognitive roles, collaborative problem-solving skills, and outcomes. Group Communication Analysis (GCA) — a computational linguistic framework for analyzing the sequential interactions of online team communication — was applied to a large CPS dataset in the domain of science (participant N = 967; team N = 480). The ETS Collaborative Science Assessment Prototype (ECSAP) was used to measure learners’ CPS skills, and CPS outcomes. Cluster analyses and linear mixed-effects modelling were used to detect learner roles, and assess the relationship between those roles on CPS skills and outcomes. Implications for future research and practice are discussed regarding sociocognitive roles and collaborative problem-solving skills.
References
Barron, B. (2000). Achieving coordination in collaborative problem-solving groups. Journal of the Learning Sciences, 9(4), 403–436. https://dx.doi.org/10.1207/S15327809JLS0904_2
Barron, B. (2003). When smart groups fail. Journal of the Learning Sciences, 12(3), 307–359. https://dx.doi.org/10.1207/S15327809JLS1203_1
Binkley, M., Erstad, O., Herman, J., Raizen, S., Ripley, M., & Rumble, M. (2011). Defining 21st century skills. In P. Griffin, B. McGaw, & E. Care (Eds.), Assessment and teaching 21st century skills (pp. 17–66). Heidelberg: Springer. https://dx.doi.org/10.1007/978-94-007-2324-5_2
Care, E., Scoular, C., & Griffin, P. (2016). Assessment of collaborative problem solving in education environments. Applied Measurement in Education, 29(4), 250–264. https://dx.doi.org/10.1080/08957347.2016.1209204
Cesareni, D., Cacciamani, S., & Fujita, N. (2015). Role taking and knowledge building in a blended university course. International Journal of Computer-Supported Collaborative Learning, 11(1), 9–39. https://dx.doi.org/10.1007/s11412- 015-9224-0
Charrad, M., Ghazzali, N., Boiteau, V., & Niknafs, A. (2014). NbClust: An R package for determining the relevant number of clusters in a data set. Journal of Statistical Software, 61(6). https://dx.doi.org/10.18637/jss.v061.i06
Chi, M. T. H. (2009). Active-Constructive-Interactive: A conceptual framework for differentiating learning activities. Topics in Cognitive Science, 1(1), 73–105. https://dx.doi.org/10.1111/j.1756-8765.2008.01005.x
Chua, S. M., Tagg, C., Sharples, M., & Rienties, B. (2017). Discussion analytics: Identifying conversations and social learners in FutureLearn MOOCs. In L. Vigentini & M. L. Urrutia (Eds.), Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 36– 62). New York: ACM. Retrieved from http://oro.open.ac.uk/id/eprint/57071
Clark, H. H. (1996). Using language. Cambridge, UK: Cambridge University Press.
Clark, H. H., & Brennan, S. E. (1991). Grounding in communication. In L. B. Resnick, J. M. Levine, & S. D. Teasley (Eds.),
Perspectives on socially shared cognition (pp. 127–149). Washington, DC: American Psychological Association.
https:// dx.doi.org/10.1037/10096-006
Dede, C. (2010). Comparing frameworks for 21st century skills. In J. Bellanca & R. Brandt (Eds.), 21st century skills: Rethinking how students learn (pp. 51–76). Bloomington, IN: Solution Tree Press.
De Wever, B., Van Keer, H., Schellens, T., & Valcke, M. (2007). Applying multilevel modelling to content analysis data: Methodological issues in the study of role assignment in asynchronous discussion groups. Learning and Instruction, 17(4), 436–447. https://dx.doi.org/10.1016/j.learninstruc.2007.04.001
Dillenbourg, P., & Fischer, F. (2007). Basics of computer-supported collaborative learning. Zeitschrift Fur Berufs-Und Wirtschaftspadagogik, 21, 111–130.
Dillenbourg, P., Järvelä, S., & Fischer, F. (2009). The evolution of research on computer-supported collaborative learning. In N. Balacheff, S. Ludvigsen, T. de Jong, A. Lazonder, & S. Barnes (Eds.), Technology-enhanced learning: Principles and products (pp. 3–19). Dordrecht: Springer Netherlands. https://dx.doi.org/10.1007/978-1-4020-9827-7_1
Dillenbourg, P., & Traum, D. (2006). Sharing solutions: Persistence and grounding in multimodal collaborative problem solving. Journal of the Learning Sciences, 15(1), 121–151. https://dx.doi.org/10.1207/s15327809jls1501_9
Doise, W. (1990). The development of individual competencies through social interaction. In H. C. Foot, M. Morgan, & R. H. Shute (Eds.), Children helping children. Chichester, UK: J. Wiley & Sons.
Dornfeld, C., & Puntambekar, S. (2015). Emergent roles and collaborative discourse over time. In O. Lindwall, P. Hakkinen, T. Koschmann, P. Tchounikine, & S. Ludvigsen (Eds.), Exploring the Material Conditions of Learning: Proceedings of the 11th International Conference on Computer Supported Collaborative Learning (CSCL 2015), 7–11 June 2015, Gothenburg, Sweden (pp. 380–387). International Society of the Learning Sciences. Retrieved from https://www.isls.org/cscl2015/papers/MC-0347-FullPaper-Dornfeld.pdf
Dowell, N. M. (2017). A computational linguistic analysis of learners’ discourse in computer-mediated group learning environments. PhD Dissertation, University of Memphis.
Dowell, N. (2019). Preparing for the future: Group communication analysis as a tool to facilitate adaptive support during digitally-mediated team interactions. Proceedings of the 3rd International Conference on AI + Adaptive Education, (AIAED). Beijing, China. New York: ACM.
Dowell, N. M., Lin, Y., Godfrey, A., Cho, H., & Brooks, C. (2019). Promoting inclusivity through time-dynamic discourse analysis in digitally-mediated collaborative learning. In S. Isotani, E. Millán, A. Ogan, P. Hastings, B. McLaren, & R. Luckin (Eds.), Proceedings of the 20th International Conference on Artificial Intelligence in Education (AIED 2019), 25–29 June 2019, Chicago, IL, USA (pp. 207–219). Springer. https://dx.doi.org/10.1007/978-3-030-23204-7_18
Dowell, N. M., Nixon, T., & Graesser, A. C. (2018). Group communication analysis: A computational linguistics approach for detecting sociocognitive roles in multi-party interactions. Behavior Research Methods, 51(3), 1007–1041. https://dx.doi.org/10.3758/s13428-018-1102-z
Dowell, N. M., Poquet, O., & Brooks, C. (2018). Applying group communication analysis to educational discourse interactions at scale. In J. Kay & R. Luckin (Eds.), Rethinking Learning in the Digital Age: Making the Learning Sciences Count. Proceedings of the 13th International Conference of the Learning Sciences (ICLS ’18), 23–27 June 2018, London, UK (3 volumes, pp. 1815–1822). International Society of the Learning Sciences. https://dx.doi.org/10.22318/cscl2018.1815
Fiore, S. M., Graesser, A., & Greiff, S. (2018). Collaborative problem-solving education for the twenty-first-century workforce. Nature Human Behaviour, 2(6), 367–369. https://dx.doi.org/10.1038/s41562-018-0363-y
Fiore, S. M., Rosen, M. A., Smith-Jentsch, K. A., Salas, E., Letsky, M., & Warner, N. (2010). Toward an understanding of macrocognition in teams: Predicting processes in complex collaborative contexts. Human Factors: The Journal of the Human Factors and Ergonomics Society, 52(2), 203–224. https://dx.doi.org/10.1177/0018720810369807
Fiore, S. M., & Schooler, J. W. (2004). Process mapping and shared cognition: Teamwork and the development of shared problem models. In E. Salas & S. M. Fiore (Eds.), Team cognition: Understanding the factors that drive process and performance (pp. 133–152). Washington, DC: American Psychological Association. https://dx.doi.org/10.1037/10690- 007
Fox, J., & Weisberg, H. S. (2010). An R companion to applied regression (2nd ed.). Thousand Oaks, CA: SAGE Publications.
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410–8415. https://dx.doi.org/10.1073/pnas.1319030111
Gervits, F., Eberhard, K., & Scheutz, M. (2016). Team communication as a collaborative process. Frontiers in Robotics and AI, 3, 62. https://dx.doi.org/10.3389/frobt.2016.00062
Gleave, E., Welser, H. T., Lento, T. M., & Smith, M. A. (2009). A conceptual and operational definition of “social role” in online community. Proceedings of the 42nd Hawaii International Conference on System Sciences (HICSS-42), 5–8 January 2009, Waikoloa, Big Island, HI, USA (pp. 1–11). IEEE Computer Society. https://dx.doi.org/10.1109/HICSS.2009.6
Graesser, A. C., Cai, Z., Morgan, B., & Wang, L. (2017). Assessment with computer agents that engage in conversational dialogues and trialogues with learners. Computers in Human Behavior, 76, 607–616. https://dx.doi.org/10.1016/j.chb.2017.03.041
Graesser, A. C., Dowell, N., & Clewley, D. (2017). Assessing collaborative problem solving through conversational agents. In A. A. Davier, M. Zhu, & P. C. Kyllonen (Eds.), Innovative assessment of collaboration (pp. 65–80). Springer. https://dx.doi.org/10.1007/978-3-319-33261-1_5
Graesser, A. C., Dowell, N. M., Hampton, A., Lippert, A. M., Li, H., & Shaffer, D. W. (2018a). Building intelligent conversational tutors and mentors for team collaborative problem solving: Guidance from the 2015 program for international student assessment. In J. J. Johnston, R. Sottilare, A. Sinatra, & C. S. Burke (Eds.), Building intelligent
tutoring systems for teams: What matters. (Vol. 19, pp. 173–214). West Yorkshire, UK: Emerald Publishing. Graesser, A. C., Fiore, S. M., Greiff, S., Andrews-Todd, J., Foltz, P. W., & Hesse, F. W. (2018b). Advancing the science of
collaborative problem solving. Psychological Science in the Public Interest, 19(2), 59–92.
https://dx.doi.org/10.1177/1529100618808244
Graesser, A. C., Foltz, P. W., Rosen, Y., Shaffer, D. W., Forsyth, C., & Germany, M. L. (2018c). Challenges of assessing collaborative problem solving. In E. Care, P. Griffin, & M. Wilson (Eds.), Assessment and teaching of 21st century skills (pp. 75–91). Springer, Cham. http://dx.doi.org/10.1007/978-3-319-65368-6_5
Greenhow, C., Robelia, B., & Hughes, J. E. (2009). Learning, teaching, and scholarship in a digital age: Web 2.0 and classroom research: What path should we take now? Educational Researcher, 38(4), 246–259.
Guo, Z., Yu, K., Pearlman, R., Navab, N., & Barmaki, R. (2019). Collaboration analysis using deep learning. arXiv [cs.HC]. Retrieved from http://arxiv.org/abs/1904.08066
Hao, J., Liu, L., von Davier, A. A., & Kyllonen, P. C. (2017). Initial steps towards a standardized assessment for collaborative problem solving (CPS): Practical challenges and strategies. In A. A. von Davier, M. Zhu, & P. C. Kyllonen (Eds.), Innovative assessment of collaboration (pp. 135–156). Springer, Cham. https://dx.doi.org/10.1007/978-3-319-33261-1_9
Hao, J., Liu, L., von Davier, A. A., Lederer, N., Zapata-Rivera, D., Jakl, P., & Bakkenson, M. (2017). EPCAL: ETS platform for collaborative assessment and learning. ETS Research Report Series, 2017(1), 1–14. https://dx.doi.org/10.1002/ets2.12181
Hao, J., Liu, L., von Davier, A., & Kyllonen, P. (2015). Assessing collaborative problem solving with simulation based tasks. In O. Lindwall, P. Hakkinen, T. Koschmann, P. Tchounikine, & S. Ludvigsen (Eds.), Exploring the Material Conditions of Learning: Proceedings of the 11th International Conference on Computer Supported Collaborative Learning (CSCL 2015), 7–11 June 2015, Gothenburg, Sweden (pp. 544–547). International Society of the Learning Sciences. Retrieved from https://pdfs.semanticscholar.org/c922/044e125a053be368e3eb146c9cf581fe55c5.pdf
Hao, J., Smith, L., Mislevy, R., von Davier, A., & Bauer, M. (2016). Taming log files from game/simulation‐based assessments: Data models and data analysis tools. ETS Research Report Series, 2016(1), 1–17. https://dx.doi.org/10.1002/ets2.12096
Hatano, G. (1993). Commentary: Time to merge Vygotskian and constructivist conceptions of knowledge acquisition. In E. A. Forman, N. Minick, & C. A. Stone (Eds.), Contexts for learning: Sociocultural dynamics in children’s development (pp. 153–166). New York: Oxford University Press.
Hecking, T., Chounta, I.-A., & Hoppe, H. U. (2016). Investigating social and semantic user roles in MOOC discussion forums. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼ16), 25–29 April 2016, Edinburgh, UK (pp. 198–207). New York: ACM. https://dx.doi.org/10.1145/2883851.2883924
Hennig, C., Meila, M., Murtagh, F., & Rocci, R. (Eds.). (2015). Handbook of cluster analysis. New York: CRC Press.
https://dx.doi.org/10.1201/b19706
Hesse, F., Care, E., Buder, J., Sassenberg, K., & Griffin, P. (2015). A framework for teachable collaborative problem solving skills. In P. Griffin & E. Care (Eds.), Assessment and teaching of 21st century skills (pp. 37–56). Springer Netherlands. Retrieved from http://link.springer.com/chapter/10.1007/978-94-017-9395-7_2
Hew, K. F., Cheung, W. S., & Ng, C. S. L. (2010). Student contribution in asynchronous online discussion: A review of the research and empirical exploration. Instructional Science, 38(6), 571–606. https://doi.org/10.1007/s11251-008-9087-0
Hmelo-Silver, C. E., & Barrows, H. S. (2008). Facilitating collaborative knowledge building. Cognition and Instruction, 26(1), 48–94. http://dx.doi.org/10.1080/07370000701798495
Hou, H.-T. (2015). Integrating cluster and sequential analysis to explore learners’ flow and behavioral patterns in a simulation game with situated-learning context for science courses: A video-based process exploration. Computers in Human Behavior, 48, 424–435. https://dx.doi.org/10.1016/j.chb.2015.02.010
Howley, I. K., & Rosé, C. P. (2016). Towards careful practices for automated linguistic analysis of group learning. Journal of Learning Analytics, 3(3), 239–262. https://dx.doi.org/10.18608/jla.2016.33.12
Hrastinski, S. (2008). What is online learner participation? A literature review. Computers & Education, 51(4), 1755–1765. https://dx.doi.org/10.1016/j.compedu.2008.05.005
Hu, X., Dowell, N., Cai, Z., Graesser, A. C., Shi, G., Cockroft, J., & Shorter, P. (2018). Constructing individual conversation characteristics curves (ICCC) for interactive intelligent tutoring environments (IITE). In R. Sottilare, A. Graesser, X. Hu, & A. Sinatra (Eds.), Design recommendations for intelligent tutoring systems, Volume 6: Team cognition. Orlando, FL: US Army Research Laboratory.
Janssen, J., Erkens, G., Kirschner, P. A., & Kanselaar, G. (2011). Multilevel analysis in CSCL research. In S. Puntambekar,
G. Erkens, & C. Hmelo-Silver (Eds.), Analyzing interactions in CSCL: Methods, approaches and issues (pp. 187–
. Boston, MA: Springer US. https://dx.doi.org/10.1007/978-1-4419-7710-6_9
Järvelä, S., Järvenoja, H., Malmberg, J., & Hadwin, A. F. (2013). Exploring socially shared regulation in the context of
collaboration. Journal of Cognitive Education and Psychology, 12, 267–286. http://dx.doi.org/10.1891/1945-
12.3.267
Järvelä, S., Volet, S., & Järvenoja, H. (2010). Research on motivation in collaborative learning: Moving beyond the cognitive–situative divide and combining individual and social processes. Educational Psychologist, 45(1), 15–27. https://dx.doi.org/10.1080/00461520903433539
Jeong, H., & Hmelo-Silver, C. E. (2016). Seven affordances of computer-supported collaborative learning: How to support collaborative learning? How can technologies help? Educational Psychologist, 51(2), 247–265. http://dx.doi.org/10.1080/00461520.2016.1158654
Kirschner, P. A., Beers, P. J., Boshuizen, H. P. A., & Gijselaers, W. H. (2008). Coercing shared knowledge in collaborative learning environments. Computers in Human Behavior, 24(2), 403–420. https://dx.doi.org/10.1016/j.chb.2007.01.028
Kittur, A., Chi, E. H., & Suh, B. (2008). Crowdsourcing user studies with Mechanical Turk. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ʼ08), 5–10 May 2008, Florence, Italy (pp. 453–456). New York: ACM. https://dx.doi.org/10.1145/1357054.1357127
Kizilcec, R. F., Piech, C., & Schneider, E. (2013). Deconstructing disengagement: Analyzing learner subpopulations in massive open online courses. Proceedings of the 3rd International Conference on Learning Analytics and Knowledge (LAK ’13), 8–12 April 2013, Leuven, Belgium (pp. 170–179). New York: ACM. https://dx.doi.org/10.1145/2460296.2460330
Klein, G., Feltovich, P. J., Bradshaw, J. M., & Woods, D. D. (2005). Common ground coordination in joint activity. In W. B. Rouse & K. R. Boff (Eds.), Organizational simulation (pp. 139–184). John Wiley & Sons. https://dx.doi.org/10.1002/0471739448.ch6
Kreijns, K., Kirschner, P. A., & Jochems, W. (2003). Identifying the pitfalls for social interaction in computer-supported collaborative learning environments: A review of the research. Computers in Human Behavior, 19(3), 335–353. https://dx.doi.org/10.1016/S0747-5632(02)00057-2
Kuhn, D. (2015). Thinking together and alone. Educational Researcher, 44(1), 46–53.
https://dx.doi.org/10.3102/0013189X15569530
Landauer, T. K., McNamara, D. S., Dennis, S., & Kintsch, W. (Eds.). (2013). Handbook of latent semantic analysis. New York: Routledge.
Lankton, N. K., McKnight, D. H., & Tripp, J. F. (2017). Facebook privacy management strategies: A cluster analysis of user privacy behaviors. Computers in Human Behavior, 76, 149–163. https://doi.org/10.1016/j.chb.2017.07.015
Lehmann-Willenbrock, N., Beck, S. J., & Kauffeld, S. (2016). Emergent team roles in organizational meetings: Identifying communication patterns via cluster analysis. Communication Studies, 67(1), 37–57. https://dx.doi.org/10.1080/10510974.2015.1074087
Letsky, M. P. (2008). Macrocognition in teams: Theories and methodologies. Aldershot, UK: Ashgate.
Lin, Y., Dowell, N. M., Godfrey, A., Cho, H., & Brooks, C. (2019). Modeling gender differences in intra- and interpersonal
dynamics during collaborative interactions. In D. Azcona & R. Chung (Eds.), Proceedings of the 9th International Conference on Learning Analytics and Knowledge (LAK ’19), 4–8 March 2019, Tempe, Arizona, USA (pp. 431–435). New York: ACM.
Liu, L., Hao, J., von Davier, A. A., Kyllonen, P., & Zapata-Rivera, J.-D. (2016). A tough nut to crack: Measuring collaborative problem solving. In Y. Rosen, S. Ferrara, & M. Mosharraf (Eds.), Handbook of research on technology tools for real-world skill development (pp. 344–359). Hershey, PA: IGI Global. https://dx.doi.org/10.4018/978-1- 4666-9441-5.ch013
Malmberg, J., Järvelä, S., & Järvenoja, H. (2017). Capturing temporal and sequential patterns of self-, co-, and socially shared regulation in the context of collaborative learning. Contemporary Educational Psychology, 49(Supplement C), 160–174. https://dx.doi.org/10.1016/j.cedpsych.2017.01.009
Mesmer-Magnus, J. R., & Dechurch, L. A. (2009). Information sharing and team performance: A meta-analysis. Journal of Applied Psychology, 94(2), 535–546. https://dx.doi.org/10.1037/a0013773
Mirriahi, N., Liaqat, D., Dawson, S., & Gašević, D. (2016). Uncovering student learning profiles with a video annotation tool: Reflective learning with and without instructional norms. Educational Technology Research and Development, 64, 1083–1106. https://dx.doi.org/10.1007/s11423-016-9449-2
Mooi, E., & Sarstedt, M. (2011). Cluster analysis. In A concise guide to market research: The process, data, and methods using IBM SPSS statistics (pp. 237–284). Springer. https://dx.doi.org/10.1007/978-3-642-12541-6
Nakagawa, S., & Schielzeth, H. (2013). A general and simple method for obtaining R2 from generalized linear mixed-effects models. Methods in Ecology and Evolution, 4(2), 133–142. https://doi.org/10.1111/j.2041-210x.2012.00261.x
National Research Council. (2011). Assessing 21st Century Skills: Summary of a Workshop. Washington, DC: National Academies Press. https://dx.doi.org/10.17226/13215
OECD. (2013). PISA 2015 collaborative problem solving framework. Oxford, UK: OECD Publishing. Piaget, J. (1993). The moral judgement of the child. New York: Simon & Schuster.
Pinheiro, J., & Bates, D. M. (2000). Mixed-effects models in S and S-Plus. Springer. Retrieved from
http://books.google.co.in/books?id=N3WeyHFbHLQC
Pinheiro, J., Bates, D., DebRoy, S., Sarkar, D., Heisterkamp, S., & Van Willigen, B. (2016). nlme: Linear and nonlinear mixed effects models (Version 3.1-128). Retrieved from https://cran.r-project.org/web/packages/nlme/index.html
Postareff, L., Mattsson, M., Lindblom-Ylänne, S., & Hailikari, T. (2017). The complex relationship between emotions, approaches to learning, study success and study progress during the transition to university. Higher Education, 73(3), 441–457. https://dx.doi.org/10.1007/s10734-016-0096-7
Preece, J., Nonnecke, B., & Andrews, D. (2004). The top five reasons for lurking: Improving community experiences for everyone. Computers in Human Behavior, 20(2), 201–223. https://dx.doi.org/10.1016/j.chb.2003.10.015
Price, E. F. (1981). Toward a taxonomy of given/new information. In P. Cole (Ed.), Radical pragmatics. New York: Academic Press.
Risser, H. S., & Bottoms, S. (2014). “Newbies” and “celebrities”: Detecting social roles in an online network of teachers via participation patterns. International Journal of Computer-Supported Collaborative Learning, 9(4), 433–450. https://dx.doi.org/10.1007/s11412-014-9197-4
Roschelle, J. (1992). Learning by collaborating: Convergent conceptual change. Journal of the Learning Sciences, 2(3), 235– 276. https://dx.doi.org/10.1207/s15327809jls0203_1
Roschelle, J., & Teasley, S. D. (1995). The construction of shared knowledge in collaborative problem-solving. In C. E. O’Malley (Ed.), Computer-supported collaborative learning (pp. 67–97). Berlin: Springer-Verlag. http://dx.doi.org/10.1007/978-3-642-85098-1_5
Rosé, C. P., Stahl, G., Goggins, S., Patterson, E., Borge, M., Carroll, J., & Duchon, A. (2014). Towards optimization of macrocognitive processes: Automating analysis of the emergence of leadership in ad hoc teams. Pittsburgh, PA: Carnegie-Mellon University.
Rosen, Y., & Foltz, P. (2014). Assessing collaborative problem solving through automated technologies. Research and Practice in Technology Enhanced Learning, 9(3), 389–410. http://dx.doi.org/10.4018/978-1-4666-5888-2.ch010
Salas, E., Cooke, N. J., & Rosen, M. A. (2008). On teams, teamwork, and team performance: Discoveries and developments. Human Factors, 50(3), 540–547. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/18689065
Scoular, C., & Care, E. (2019). Monitoring patterns of social and cognitive student behaviors in online collaborative problem solving assessments. Computers in Human Behavior, 104. https://dx.doi.org/10.1016/j.chb.2019.01.007
Scoular, C., Care, E., & Hesse, F. W. (2017). Designs for operationalizing collaborative problem solving for automated assessment: Operationalizing collaborative problem solving. Journal of Educational Measurement, 54(1), 12–35. https://dx.doi.org/10.1111/jedm.12130
Siddiq, F., & Scherer, R. (2017). Revealing the processes of students’ interaction with a novel collaborative problem solving task: An in-depth analysis of think-aloud protocols. Computers in Human Behavior, 76, 509–525. https://dx.doi.org/10.1016/j.chb.2017.08.007
Singley, M. K., Fairweather, P. G., & Swerling, S. (1999). Team tutoring systems: Reifying roles in problem solving. In C. Hoadley & J. Roschelle (Eds.), Proceedings of the 1999 Conference on Computer-Supported Collaborative Learning (CSCL ’99), 12–15 December 1999, Palo Alto, CA, USA (pp. 66–es). International Society of the Learning Sciences. Retrieved from http://dl.acm.org/citation.cfm?id=1150240.1150306
Spada, H. (2010). Of scripts, roles, positions, and models. Computers in Human Behavior, 26(4), 547–550. https://dx.doi.org/10.1016/j.chb.2009.08.011
Stahl, G. (2002). Rediscovering CSCL. In T. Koschmann, R. Hall, & N. Miyake (Eds.), CSCL 2: Carrying forward the conversation (pp. 169–181). Hillsdale, NJ: Lawrence Erlbaum. Retrieved from https://pdfs.semanticscholar.org/e717/72b208cdc16c2851eaa287e58be9bec19d22.pdf?_ga=2.79495700.1684383047.1 580443850-1453764810.1580443850
Stahl, G., & Rosé, C. P. (2013). Theories of team cognition: Cross-disciplinary perspectives. In E. Salas, S. M. Fiore, & M. P. Letsky (Eds.), Theories of team cognition: Cross-disciplinary perspectives (pp. 111–134). London: Routledge.
Strijbos, J.-W., & Weinberger, A. (2010). Emerging and scripted roles in computer-supported collaborative learning. Computers in Human Behavior, 26(4), 491–494. https://dx.doi.org/10.1016/j.chb.2009.08.006
Subbalakshmi, C., Krishna, G. R., Rao, S. K. M., & Rao, P. V. (2015). A method to find optimum number of clusters based on fuzzy silhouette on dynamic data set. Procedia Computer Science, 46, 346–353. https://dx.doi.org/10.1016/j.procs.2015.02.030
Suthers, D. D., Dwyer, N., Medina, R., & Vatrapu, R. (2010). A framework for conceptualizing, representing, and analyzing distributed interaction. International Journal of Computer-Supported Collaborative Learning, 5(1), 5–42.
Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston, MA: Allyn & Bacon/Pearson Education.
Taniar, T. (2006). Research and trends in data mining technologies and applications. Hershey, PA: IGI Global. Retrieved from https://books.google.com/books?id=17bSnt3DGUYC
Teasley, S. D., Fischer, F., Weinberger, A., Stegmann, K., Dillenbourg, P., Kapur, M., & Chi, M. (2008). Cognitive convergence in collaborative learning. Proceedings of the 8th International Conference on the Learning Sciences (ICLS ’08), 24–28 June 2008, Utrecht, Netherlands (Vol. 3, pp. 630–637). International Society of the Learning Sciences.
Teasley, S. D., & Roschelle, J. (1993). Constructing a joint problem space: The computer as a tool for sharing knowledge. In S. P. Lajoie & S. L. Derry (Eds.), Computers as cognitive tools (pp. 229–258). Hillsdale, NJ: Erlbaum.
Van Boxtel, C. (2004). Studying peer interaction from three perspectives. In J. Linden & P. Renshaw (Eds.), Dialogic learning (pp. 125–143). New York: Springer. Retrieved from http://link.springer.com/chapter/10.1007/1-4020-1931- 9_7
van den Bossche, P., Segers, M., & Kirschner, P. A. (2006). Social and cognitive factors driving teamwork in collaborative learning environments: Team learning beliefs and behaviors. Small Group Research, 37(5), 490–521. Retrieved from https://dspace.library.uu.nl/handle/1874/16978
von Davier, A. A., & Halpin, P. F. (2013). Collaborative problem solving and the assessment of cognitive skills: Psychometric considerations. ETS Research Report Series, 2013(2), i-36. https://doi.org/10.1002/j.2333- 8504.2013.tb02348.x
von Davier, A. A., Hao, J., Liu, L., & Kyllonen, P. (2017). Interdisciplinary research agenda in support of assessment of collaborative problem solving: Lessons learned from developing a collaborative science assessment prototype. Computers in Human Behavior, 76, 631–640. https://dx.doi.org/10.1016/j.chb.2017.04.059
Voogt, J., Erstad, O., Dede, C., & Mishra, P. (2013). Challenges to learning and schooling in the digital networked world of the 21st century. Journal of Computer Assisted Learning, 29(5), 403–413. doi:10.1111/jcal.12029
Wise, A. F., & Schwarz, B. B. (2017). Visions of CSCL: Eight provocations for the future of the field. International Journal of Computer-Supported Collaborative Learning, 12(4), 423–467. https://dx.doi.org/10.1007/s11412-017-9267-5
Wise, A. F., Speer, J., Marbouti, F., & Hsiao, Y.-T. (2012). Broadening the notion of participation in online discussions: Examining patterns in learners’ online listening behaviors. Instructional Science, 41(2), 323–343. https://dx.doi.org/10.1007/s11251-012-9230-9
Yeh, Y. C. (2010). Analyzing online behaviors, roles, and learning communities via online discussions. Journal of Educational Technology & Society, 13(1), 140–151. Retrieved from http://www.jstor.org/stable/jeductechsoci.13.1.140
Zapata-Rivera, D., Jackson, T., Liu, L., Bertling, M., Vezzu, M., & Katz, I. R. (2014). Assessing science inquiry skills using trialogues. In S. Trausan-Matu, K. E. Boyer, M. Crosby, & K. Panourgia (Eds.), Proceedings of the 12th International Conference on Intelligent Tutoring Systems (ITS 2014), 5–9 June 2014, Honolulu, HI, USA (pp. 625–626). Springer. https://dx.doi.org/10.1007/978-3-319-07221-0_84
Downloads
Published
How to Cite
Issue
Section
License
TEST
