Measuring the Impact of Interdependence on Individuals During Collaborative Problem-Solving
DOI:
https://doi.org/10.18608/jla.2021.7240Keywords:
collaborative problem-solving, collaboration analytics, interdependence, epistemic network analysisAbstract
Collaboration analytics often focuses on assessing and monitoring individuals during collaborative problem-solving (CPS). A defining feature of CPS is the interdependence that exists between individuals when they work together — that is, how they respond to and influence one another over time. While models that account for the impact of interdependence at the individual level of analysis (interdependent models) exist, they are often highly complex.
This complexity makes them potentially difficult to use in assessments and systems that need to be explainable for educators, learners, and other researchers. Measures of the impact of interdependence at the individual level of analysis could inform decisions as to whether interdependent models should be used, or whether simpler models will suffice. Such measures could also be used to investigate specific questions about interdependence in collaborative settings. In this paper, I present a novel method of measuring the impact of interdependence on individuals using epistemic network analysis. To provide evidence of the validity of the measure, I compare it to qualitative findings that describe the impact of interdependence on individuals participating in team training scenarios. To demonstrate the value of the measure, I use it to assess the impact of interdependence in these data overall and to test hypotheses regarding the collaborative task design. My results suggest that the measure can distinguish between individuals who have been impacted by interdependence differently, that interdependence is impactful in these data overall, and that aspects of the task design may have affected how some individuals were impacted by interdependence.
References
Aronson, E. (1978). The Jigsaw Classroom. Newbury Park, CA: Sage.
Bakeman, R., & Gottman, J. M. (1997). Observing Interaction: An Introduction to Sequential Analysis. New York: Cambridge University Press. https://doi.org/10.1017/CBO9780511527685
Burnham, K. P., & Anderson, D. R. (2004). Multimodel inference: Understanding AIC and BIC in model selection. Sociological Methods & Research, 33(2), 261–304. https://doi.org/10.1177/0049124104268644
Chi, M. T. (1997). Quantifying qualitative analyses of verbal data: A practical guide. The Journal of the Learning Sciences, 6(3), 271–315. https://doi.org/10.1207/s15327809jls06031
Chinn, C. A., & Sherin, B. (2014). Microgenetic methods. In R. K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences (pp. 171–190). Cambridge University Press. https://doi.org/10.1017/CBO9781139519526.012
Clark, H. H. (1996). Using Language. Cambridge University Press. https://doi.org/10.1017/CBO9780511620539
Csanadi, A., Eagan, B., Kollar, I., Shaffer, D. W., & Fischer, F. (2018). When coding-and-counting is not enough: Using epistemic network analysis (ENA) to analyze verbal data in CSCL research. International Journal of Computer-Supported Collaborative Learning, 13(4), 419–438. https://doi.org/10.1007/s11412-018-9292-z
Dascalu, M., McNamara, D. S., Trausan-Matu, S., & Allen, L. K. (2018, April). Cohesion network analysis of CSCL participation. Behavior Research Methods, 50(2), 604–619. https://doi.org/10.3758/s13428-017-0888-4
DeChurch, L. A., & Mesmer-Magnus, J. R. (2010). The cognitive underpinnings of effective teamwork: A meta-analysis. Journal of Applied Psychology, 95(1), 32–53. https://doi.org/10.1037/a0017328
Dillenbourg, P. (1999). Collaborative Learning: Cognitive and Computational Approaches. Advances in Learning and Instruction Series. ERIC. https://doi.org/10.1016/S0360-1315(00)00011-7
Dowell, N. M. M., Nixon, T. M., & Graesser, A. C. (2019, June). Group communication analysis: A computational linguistics approach for detecting sociocognitive roles in multiparty interactions. Behavior Research Methods, 51(3), 1007–1041. https://doi.org/10.3758/s13428-018-1102-z
Fiore, S. M., Graesser, A. C., Greiff, S., Griffin, P., Gong, B., Kyllonen, P. C., . . . von Davier, A. (2017). Collaborative problem solving: Considerations for the national assessment of educational progress. Retrieved from https://orbilu.uni.lu/handle/10993/31897
Forsyth, D. R. (2009). Group Dynamics. Cengage Learning.
Gee, J. P. (1999). An Introduction to Discourse Analysis: Theory and Method. London, UK: Routledge. https://doi.org/10.4324/9780203005675
Glaser, B. G., & Strauss, A. L. (1967). The Discovery of Grounded Theory: Strategies for Qualitative Research. Aldine de Gruyter. https://doi.org/10.4324/9780203793206-1
Griffin, P., & Care, E. (2014). Assessment and Teaching of 21st Century Skills: Methods and Approach. Springer. https://doi.org/10.1007/978-94-017-9395-7
Gweon, G., Jain, M., McDonough, J., Raj, B., & Ros´e, C. P. (2013). Measuring prevalence of other-oriented transactive contributions using an automated measure of speech style accommodation. International Journal of Computer-Supported Collaborative Learning, 8(2), 245–265. https://doi.org/10.1007/s11412-013-9172-5
Halpin, P. F., & von Davier, A. A. (2017). Modeling collaboration using point processes. In A. von Davier, M. Zhu, & P. Kyllonen (Eds.), Innovative Assessment of Collaboration (pp. 233–247). Springer. https://doi.org/10.1007/978-3-319-33261-115
Hutchins, E. (1995). Cognition in the Wild. Cambridge, MA, USA: MIT Press. https://doi.org/10.1023/A:1008285430235
Janssen, J., Erkens, G., Kirschner, P. A., & Kanselaar, G. (2011). Multilevel analysis in CSCL research. In S. Puntambekar, G. Erkens, & C. Hmelo-Silver (Eds.), Analyzing Interactions in CSCL (pp. 187–205). Springer. https://doi.org/10.1007/978-1-4419-7710-69
Johnston, J. H., Poirier, J., & Smith-Jentsch, K. A. (1998). Decision making under stress: Creating a research methodology. In J. A. Cannon-Bowers & E. Salas (Eds.), Making Decisions under Stress: Implications for Individual and Team Training (pp. 39–59). Washington, D.C.: American Psychological Association. https://doi.org/10.1037/10278-002
Kane, M. (2006). Validation. In R. Brennan (Ed.), Educational Measurement (4th ed., pp. 17–64). Westport, CT: American Council on Education and Praeger.
Kapur, M. (2011, March). Temporality matters: Advancing a method for analyzing problem-solving processes in a computer-supported collaborative environment. International Journal of Computer-Supported Collaborative Learning, 6(1), 39–56. https://doi.org/10.1007/s11412-011-9109-9
Kozlowski, S. W., & Ilgen, D. R. (2006). Enhancing the effectiveness of work groups and teams. Psychological Science in the Public Interest, 7(3), 77–124. https://doi.org/10.1111/j.1529-1006.2006.00030.x
Landauer, T. K., Foltz, P. W., & Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 25(2-3), 259–284. https://doi.org/10.1080/01638539809545028
Marks, M. A., Mathieu, J. E., & Zaccaro, S. J. (2001). A temporally based framework and taxonomy of team processes. The Academy of Management Review, 26(3), 356. https://doi.org/10.2307/259182
Marquart, C. L., Swiecki, Z., Collier, W., Eagan, B., Woodward, R., & Shaffer, D. W. (2018). rENA: Epistemic Network Analysis. Retrieved from https://cran.r-project.org/web/packages/rENA/index.html
Marquart, C. L., Swiecki, Z., Eagan, B., & Shaffer, D. W. (2018). ncodeR. Retrieved from https://cran.r-project.org/web/packages/ncodeR/ncodeR.pdf
Martinez-Maldonado, R., Kay, J., Buckingham Shum, S., & Yacef, K. (2019). Collocated collaboration analytics: Principles and dilemmas for mining multimodal interaction data. Human–Computer Interaction, 34(1), 1–50. https://doi.org/10.1080/07370024.2017.1338956
Mathieu, J. E., Hollenbeck, J. R., van Knippenberg, D., & Ilgen, D. R. (2017). A century of work teams in the Journal of Applied Psychology. Journal of Applied Psychology, 102(3), 452–467. https://doi.org/10.1037/apl0000128
Miyake, N., & Kirschner, P. A. (2014). The social and interactive dimensions of collaborative learning. In R. K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences (pp. 418–438). New York: Cambridge University Press. https://doi.org/10.1017/CBO9781139519526.026
OECD. (2017). PISA 2015 Assessment and Analytical Framework: Science, Reading, Mathematic, Financial Literacy and Collaborative Problem Solving, revised edition. Paris: PISA, OECD Publishing. https://doi.org/10.1787/9789264281820-en
Olson, G. M., Herbsleb, J. D., & Rueter, H. H. (1994). Characterizing the sequential structure of interactive behaviors through statistical and grammatical techniques. Human-Computer Interaction, 9(3-4), 427–472. Retrieved from https://dl.acm.org/doi/10.1207/s15327051hci0903%252646
Reimann, P. (2009). Time is precious: Variable- and event-centred approaches to process analysis in CSCL research. International Journal of Computer-Supported Collaborative Learning, 4(3), 239–257. https://doi.org/10.1007/s11412-009-9070-z
Roschelle, J. (1992). Learning by collaborating: Convergent conceptual change. The Journal of the Learning Sciences, 2(3), 235–276. https://doi.org/10.1207/s15327809jls02031
Ruis, A. R., Siebert-Evenstone, A. L., Pozen, R., Eagan, B. R., & Shaffer, D. W. (2019). Finding common ground: A method for measuring recent temporal context in analyses of complex, collaborative thinking. In K. Lund, G. P. Niccolai, C. Hmelo-Silver, G. Gwon, & M. Baker (Eds.), A Wide Lens: Combining Embodied, Enactive, Extended, and Embedded Learning in Collaborative Settings: 13th International Conference on Computer Supported Collaborative Learning (CSCL ’19), 17–21 June 2019, Lyon, France (Vol. 1, pp. 136–143). Retrieved from https://repository.isls.org/handle/1/1559
Rummel, N., Walker, E., & Aleven, V. (2016, June). Different futures of adaptive collaborative learning support. International Journal of Artificial Intelligence in Education, 26(2), 784–795. https://doi.org/10.1007/s40593-016-0102-3
Schrodinger, E. (1935). Die gegenw¨artige Situation in der Quantenmechanik. Naturwissenschaften, 23(48), 807–812. https://doi.org/10.1007/BF01491987
Shaffer, D. W. (2017). Quantitative Ethnography. Madison, WI: Cathcart Press.
Shaffer, D. W., Collier, W., & Ruis, A. R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3(3), 9–45. https://doi.org/10.18608/jla.2016.33.3
Siebert-Evenstone, A. L., Irgens, G. A., Collier, W., Swiecki, Z., Ruis, A. R., & Shaffer, D. W. (2017). In search of conversational grain size: Modelling semantic structure using moving stanza windows. Journal of Learning Analytics, 4(3), 123–139. https://doi.org/10.18608/jla.2017.43.7
Snijders, T. A. B., & Bosker, R. J. (2012). Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling. Thousand Oaks, CA: Sage.
Stahl, G., Law, N., Cress, U., & Ludvigsen, S. (2014). Analyzing roles of individuals in small-group collaboration processes. International Journal of Computer-Supported Collaborative Learning, 9(4), 365–370. https://doi.org/10.1007/s11412-014-9204-9
Stylianou-Georgiou, A., Pananastiou, E., & Puntambekar, S. (2011). Analyzing collaborative processes and learning from hypertext through hierarchical linear modeling. In S. Puntambekar, G. Erkens, & C. Hmelo-Silver (Eds.), Analyzing Interactions in CSCL (pp. 145–159). Boston, MA: Springer. https://doi.org/10.1007/978-1-4419-7710-67
Suthers, D. D. (2017). Multilevel analysis of activity and actors in heterogeneous networked learning environments. In C. Lang, G. Siemens, A. Wise, & D. Gasevic (Eds.), Handbook of Learning Analytics (1st ed., pp. 189–197). Society for Learning Analytics Research (SoLAR). https://doi.org/10.18608/hla17.016
Suthers, D. D., & Desiato, C. (2012). Exposing chat features through analysis of uptake between contributions. In Proceedings of the Hawaii International Conference on System Sciences (HICSS ’12), 4–7 January 2012, Honolulu, HI, USA (pp. 3368–3377). IEEE. https://doi.org/10.1109/HICSS.2012.274
Swiecki, Z., Ruis, A. R., Farrell, C., & Shaffer, D. W. (2020). Assessing individual contributions to collaborative problem solving: A network analysis approach. Computers in Human Behavior, 104. https://doi.org/10.1016/j.chb.2019.01.009
Van de Ven, A. H., Delbecq, A. L., & Koenig Jr., R. (1976). Determinants of coordination modes within organizations. American Sociological Review, 41(2), 322–338. https://doi.org/10.2307/2094477
Wang, X., Wen, M., & Ros´e, C. (2017). Contrasting explicit and implicit scaffolding for transactive exchange in team oriented project based learning. In B. K. Smith, M. Borge, E. Mercier, & K. Y. Lim (Eds.), Making a Difference: Prioritizing Equity and Access in CSCL, 12th International Conference on Computer Supported Collaborative Learning (CSCL ’17), 18–22 June 2017, Philadelphia, PA, USA (Vol. 1, pp. 25–32). International Society of the Learning Sciences. Retrieved from https://repository.isls.org/handle/1/274
Webb, N. M. (2008). Teacher Practices and Small-Group Dynamics in Cooperative Learning Classrooms. In R. Gillies, A. Ashman, & J. Terwel (Eds.), The teacher’s role in implementing cooperative learning in the classroom (pp. 201–221). Springer. https://doi.org/10.1007/978-0-387-70892-810
Witten, I. H., Frank, E., Hall, M. A., & Pal, C. J. (2016). Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann. https://doi.org/10.1016/C2009-0-19715-5Dascalu, M., McNamara, D. S., Trausan-Matu, S., & Allen, L. K. (2018, April). Cohesion network analysis of CSCL participation. Behavior Research Methods, 50(2), 604–619. https://doi.org/10.3758/s13428-017-0888-4
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2021 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
