What Do You Mean by Collaboration Analytics? A Conceptual Model

Roberto Martinez-Maldonado; Dragan Gašević; Vanessa Echeverria; Gloria Fernandez Nieto; Zachari Swiecki; Simon Buckingham Shum

doi:10.18608/jla.2021.7227

Authors

Roberto Martinez-Maldonado Monash University https://orcid.org/0000-0002-8375-1816
Dragan Gašević Monash University https://orcid.org/0000-0001-9265-1908
Vanessa Echeverria Carnegie Mellon University https://orcid.org/0000-0002-2022-9588
Gloria Fernandez Nieto University of Technology Sydney https://orcid.org/0000-0002-8163-2303
Zachari Swiecki University of Wisconsin-Madison https://orcid.org/0000-0002-7414-5507
Simon Buckingham Shum University of Technology Sydney https://orcid.org/0000-0002-6334-7429

DOI:

https://doi.org/10.18608/jla.2021.7227

Keywords:

collaborative learning, teamwork, CSCL, CSCW, multimodal learning analytics, group work, learning design

Abstract

Using data to generate a deeper understanding of collaborative learning is not new, but automatically analyzing log data has enabled new means of identifying key indicators of effective collaboration and teamwork that can be used to predict outcomes and personalize feedback. Collaboration analytics is emerging as a new term to refer to computational methods for identifying salient aspects of collaboration from multiple group data sources for learners, educators, or other stakeholders to gain and act upon insights. Yet, it remains unclear how collaboration analytics go beyond previous work focused on modelling group interactions for the purpose of adapting instruction. This paper provides a conceptual model of collaboration analytics to help researchers and designers identify the opportunities enabled by such innovations to advance knowledge in, and provide enhanced support for, collaborative learning and teamwork. We argue that mapping from low-level data to higher-order constructs that are educationally meaningful, and that can be understood by educators and learners, is essential to assessing the validity of collaboration analytics. Through four cases, the paper illustrates the critical role of theory, task design, and human factors in the design of interfaces that inform actionable insights for improving collaboration and group learning.

References

Ahn, J., Campos, F., Hays, M., & DiGiacomo, D. (2019). Designing in context: Reaching beyond usability in learning analytics dashboard design. Journal of Learning Analytics, 6(2), 70–85. https://doi.org/10.18608/jla.2019.62.5

An, P., Holstein, K., d’Anjou, B., Eggen, B., & Bakker, S. (2020). The TA framework: Designing real-time teaching augmentation for K–12 Classrooms. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’20), 25–30 April 2020, Honolulu, HI, USA (pp. 1–17). New York: ACM. https://doi.org/10.1145/3313831.3376277

Anaya, A. R., González-Boticario, J., Letón, E., & Hernández-del-Olmo, F. (2015). An approach of collaboration analytics in MOOCs using social network analysis and influence diagrams. Proceedings of the Eighth International Conference on Educational Data Mining (EDM ’15), 26–29 June 2015, Madrid, Spain (pp. 492–495). https://www.educationaldatamining.org/EDM2015/proceedings/short492-495.pdf

ANZCOR. (2016). ARC Resuscitation Guidelines. Melbourne, Australia: Australian and New Zealand Resuscitation Council. Retrieved from https://resus.org.au/guidelines/

Aviv, R., Erlich, Z., & Ravid, G. (2003). Cohesion and roles: Network analysis of CSCL communities. Proceedings of the IEEE International Conference on Advanced Technologies (ICAT ’03), 9–11 July 2003, Athens, Greece (pp. 145–149). IEEE. https://doi.org/10.1109/ICALT.2003.1215045

Bae, H., Glazewski, K., Hmelo-Silver, C., Lester, J., Mott, B. W., & Rowe, J. (2019). Intelligent cognitive assistants to support orchestration in CSCL. Proceedings of the International Conference on Computer Supported Collaborative Learning (CSCL ’19), 17–21 June 2019, Lyon, France (pp. 947–948). ISLS. https://repository.isls.org/handle/1/1743

Bakharia, A., Corrin, L., de Barba, P., Kennedy, G., Gašević, D., Mulder, R., . . . Lockyer, L. (2016). A conceptual framework linking learning design with learning analytics. Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK ’16), 25–29 April 2016, Edinburgh, Scotland (pp. 329–338). New York: ACM. https://doi.org/10.1145/2883851.2883944

Behrens, J. T., DiCerbo, K. E., & Foltz, P. W. (2019). Assessment of complex performances in digital environments. The Annals of the American Academy of Political and Social Science, 683(1), 217–232. https://doi.org/10.1177/0002716219846850

Buckingham Shum, S., & Crick, R. D. (2016). Learning analytics for 21st century competencies. Journal of Learning Analytics, 3(2), 6–21. https://doi.org/10.18608/jla.2016.32.2

Buckingham Shum, S., Echeverria, V., & Martinez-Maldonado, R. (2019a). The multimodal matrix as a quantitative ethnography methodology. Proceedings of the International Conference on Quantitative Ethnography (ICQE ’19), 19–22 October 2019, Madison, WI, USA (pp. 26–40). Springer. https://doi.org/10.1007/978-3-030-33232-7_3

Buckingham Shum, S., & Ferguson, R. (2012). Social learning analytics. Journal of Educational Technology & Society, 15(3), 3–26. https://www.jstor.org/stable/jeductechsoci.15.3.3

Buckingham Shum, S., Ferguson, R., & Martinez-Maldonado, R. (2019b). Human-centred learning analytics. Journal of Learning Analytics, 6(2), 1–9. https://doi.org/10.18608/jla.2019.62.1

Buckingham Shum, S., & McKay, T. A. (2018). Architecting for learning analytics: Innovating for sustainable impact. EDUCAUSE Review, March/April 2018, 25–37. https://er.educause.edu/articles/2018/3/architecting-for-learning-analytics-innovating-for-sustainable-impact

Bull, S., & Vatrapu, R. (2011). Supporting collaborative interaction with open learner models: Existing approaches and open questions. Proceedings of the Ninth International Conference on Computer Supported Collaborative Learning 2011 (CSCL ’11), 4–8 July 2011, Hong Kong, China (pp. 761–765). ISLS.

Chandrasegaran, S., Bryan, C., Shidara, H., Chuang, T.-Y., & Ma, K.-L. (2019). TalkTraces: Real-time capture and visualization of verbal content in meetings. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’19), 4–9 May 2019, Glasgow, UK (pp. 577:571–577:514). New York: ACM. https://doi.org/10.1145/3290605.3300807

Chen, J., Wang, M., Kirschner, P. A., & Tsai, C.-C. (2018). The role of collaboration, computer use, learning environments, and supporting strategies in CSCL: A meta-analysis. Review of Educational Research, 88(6), 799–843. https://doi.org/10.3102/0034654318791584

Choudhury, T., & Pentland, A. (2002). The sociometer: A wearable device for understanding human networks. Proceedings of the Workshop: Ad Hoc Communications and Collaboration in Ubiquitous Computing Environments at CSCW ’02, 16–20 November 2002, New Orleans, USA (pp. 1–6). MIT. https://alumni.media.mit.edu/~tanzeem/shortcuts/workingpaper.pdf

Clow, D. (2012). The learning analytics cycle: closing the loop effectively. Proceedings of the Second International Conference on Learning Analytics & Knowledge (LAK ’12), 29 April–2 May 2012, Vancouver, Canada (pp. 134–138). New York: ACM. https://doi.org/10.1145/2330601.2330636

Clow, D. (2014). Data wranglers: Human interpreters to help close the feedback loop. Proceedings of the Fourth International Conference on Learning Analytics & Knowledge (LAK ’14), 24–28 March 2014, Indianapolis, USA (pp. 49–53). New York: ACM. https://doi.org/10.1145/2567574.2567603

Colvin, C., Rogers, T., Wade, A., Dawson, S., Gasevic, D., Buckingham Shum, S., . . . Kennedy, G. (2016). Student Retention and Learning Analytics: A Snapshot of Australian Practices and a Framework for Advancement. Australian Government Office for Learning and Teaching (Report), University of South Australia. Retrieved from https://opus.lib.uts.edu.au/handle/10453/117173

Cooper, A. (2012). What is analytics? Definition and essential characteristics. CETIS Analytics Series, 1(5), 1–10. https://publications.cetis.org.uk/2012/521

Corrin, L., & De Barba, P. (2015). How do students interpret feedback delivered via dashboards? Proceedings of the Fifth International Conference on Learning Analytics & Knowledge (LAK ’15), 16–20 March 2015, Poughkeepsie, New York, USA (pp. 430–431). New York: ACM. https://doi.org/10.1145/2723576.2723662

Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281. https://doi.org/10.1037/h0040957

Davenport, T. H., & Harris, J. G. (2007). Competing on Analytics: The New Science of Winning (Vol. 84). Cambridge, MA, USA: Harvard Business School Press.

Dawson, S., Mirriahi, N., & Gasevic, D. (2015). Importance of theory in learning analytics in formal and workplace settings. Journal of Learning Analytics, 2(2), 1–4. https://doi.org/10.18608/jla.2015.22.1

Day, D. V., Gronn, P., & Salas, E. (2004). Leadership capacity in teams. The Leadership Quarterly, 15(6), 857–880. https://doi.org/10.1016/j.leaqua.2004.09.001

De Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M., & Cannavacciuolo, L. (2011). Discourse-centric learning analytics. Proceedings of the First International Conference on Learning Analytics & Knowledge (LAK ’11), 27 February–1 March 2011, Banff, AB, Canada (pp. 23–33). New York: ACM. https://doi.org/10.1145/2090116.2090120

Dillenbourg, P. (1998). What do you mean by “collaborative learning”? In P. Dillenbourg (Ed.), Collaborative Learning: Cognitive and Computational Approaches. Advances in Learning and Instruction Series (pp. 1–19). Oxford: Elsevier Science.

Dillenbourg, P. (1999). Collaborative Learning: Cognitive and Computational Approaches. Advances in Learning and Instruction Series. New York: Elsevier.

Dillenbourg, P., & Baker, M. (1996). Negotiation spaces in human-computer collaborative learning. Proceedings of the International Conference on Design of Cooperative Systems (COOP ’96), 12–14 June 1996, Juan-Les-Pins, France (pp. 187–206). EUSSET.

Dollinger, M., Liu, D., Arthars, N., & Lodge, J. (2019). Working together in learning analytics towards the co-creation of value. Journal of Learning Analytics, 6(2), 10–26. https://doi.org/10.18608/jla.2019.62.2

Dönmez, P., Rosé, C., Stegmann, K., Weinberger, A., & Fischer, F. (2005). Supporting CSCL with automatic corpus analysis technology. Proceedings of the International Conference on Computer-Support for Collaborative Learning, (CSCL ’05), 30 May–4 June 2005, Taipei, Taiwan (pp. 125–134). ISLS. https://dl.acm.org/doi/10.5555/1149293.1149310

Dowell, N. M. M., Nixon, T. M., & Graesser, A. C. (2019). Group communication analysis: A computational linguistics approach for detecting sociocognitive roles in multiparty interactions. Behavior Research Methods, 51(3), 1007–1041. https://doi.org/10.3758/s13428-018-1102-z

Drachsler, H., Hoel, T., Scheffel, M., Kismihók, G., Berg, A., Ferguson, R., . . . Manderveld, J. (2015). Ethical and privacy issues in the application of learning analytics. Proceedings of the Fifth International Conference on Learning Analytics & Knowledge (LAK ’15), 16–20 March 2015, Poughkeepsie, New York, USA (pp. 390–391). New York: ACM. https://doi.org/10.1145/2723576.2723642

Echaluce, M. L. S., Fidalgo-Blanco, A., Esteban-Escano, J., Peñalvo, F. J. G., & González, M. Á. C. (2018). Using learning analytics to detect authentic leadership characteristics in engineering students. The International Journal of Engineering Education, 34(3), 851–864. http://repositorio.grial.eu/handle/grial/1582

Echeverria, V. (2020). Designing and Validating Automated Feed-back for Collocated Teams Using Multimodal Learning Analytics (PhD in Learning Analytics). University of Technology Sydney (UTS), Sydney, Australia.

Echeverria, V., Martinez-Maldonado, R., & Buckingham Shum, S. (2019). Towards collaboration translucence: Giving meaning to multimodal group data. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’19), 4–9 May 2019, Glasgow, UK (paper number 39, pp. 1–16). New York: ACM. https://doi.org/10.1145/3290605.3300269

Echeverria, V., Martinez-Maldonado, R., Buckingham Shum, S., Chiluiza, K., Granda, R., & Conati, C. (2018a). Exploratory versus explanatory visual learning analytics: Driving teachers’ attention through educational data storytelling. Journal of Learning Analytics, 5(3), 72–97. https://doi.org/10.18608/jla.2018.53.6

Echeverria, V., Martinez-Maldonado, R., Granda, R., Chiluiza, K., Conati, C., & Buckingham Shum, S. (2018b). Driving data storytelling from learning design. Proceedings of the Eighth International Conference on Learning Analytics and Knowledge (LAK ’18), 5–9 March 2018, Sydney, Australia (pp. 131–140). New York: ACM. https://doi.org/10.1145/3170358.3170380

Echeverria, V., Martinez-Maldonado, R., Power, T., Hayes, C., & Buckingham Shum, S. (2018c). Where is the nurse? Towards automatically visualising meaningful team movement in healthcare education. Proceedings of the International Conference on Artificial Intelligence in Education (AIED ’18), 27–30 June 2018, London, UK (pp. 74–78). Springer. https://doi.org/10.1007/978-3-319-93846-2_14

Ellis, C. A., Gibbs, S. J., & Rein, G. (1991). Groupware: Some issues and experiences. Communications of the ACM, 34(1), 39–58. https://doi.org/10.1145/99977.99987

Ferreira, M., Rolim, V., Mello, R. F., Lins, R. D., Chen, G., & Gašević, D. (2020). Towards automatic content analysis of social presence in transcripts of online discussions. Proceedings of the 10th International Conference on Learning Analytics & Knowledge (LAK ’20), 23–27 March 2020, Frankfurt, Germany (pp. 141–150). New York: ACM. https://doi.org/10.1145/3375462.3375495

Fiore, S. (2019). Integrating theorizing on embodied, enactive, extended, and embedded cognition to augment CSCL research. Keynote at the International Conference of Computer-Supported Collaborative Learning (CSCL ’19), 17–21 June 2019, Lyon, France. https://www.cscl2019.com/en/keynotes-broadcast/51.

Fiore, S. M. (2008). Interdisciplinarity as teamwork: How the science of teams can inform team science. Small Group Research, 39(3), 251–277. https://doi.org/10.1177/1046496408317797

Fitzpatrick, G., & Ellingsen, G. (2013). A review of 25 years of CSCW research in healthcare: Contributions, challenges and future agendas. Computer Supported Cooperative Work, 22(4-6), 609–665. https://doi.org/10.1007/s10606-012-9168-0

Fransen, J., Weinberger, A., & Kirschner, P. A. (2013). Team effectiveness and team development in CSCL. Educational Psychologist, 48(1), 9–24. https://doi.org/10.1080/00461520.2012.747947

Garrison, D. R., & Arbaugh, J. B. (2007). Researching the community of inquiry framework: Review, issues, and future directions. The Internet and Higher Education, 10(3), 157–172. https://doi.org/10.1016/j.iheduc.2007.04.001

Gašević, D., Adesope, O., Joksimović, S., & Kovanović, V. (2015a). Externally-facilitated regulation scaffolding and role assignment to develop cognitive presence in asynchronous online discussions. The Internet and Higher Education, 24, 53–65. https://doi.org/10.1016/j.iheduc.2014.09.006

Gašević, D., Dawson, S., Rogers, T., & Gasevic, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education, 28, 68–84. https://doi.org/10.1016/j.iheduc.2015.10.002

Gašević, D., Dawson, S., & Siemens, G. (2015b). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1), 64–71. https://doi.org/10.1007/s11528-014-0822-x

Gašević, D., Joksimović, S., Eagan, B. R., & Shaffer, D. W. (2018). SENS: Network analytics to combine social and cognitive perspectives of collaborative learning. Computers in Human Behavior, 92(March 2019), 562–577. https://doi.org/10.1016/j.chb.2018.07.003

Gašević, D., Kovanović, V., & Joksimović, S. (2017). Piecing the learning analytics puzzle: A consolidated model of a field of research and practice. Learning: Research and Practice, 3(1), 63–78. https://doi.org/10.1080/23735082.2017.1286142

Gibson, A., & Martinez-Maldonado, R. (2017). That dashboard looks nice, but what does it mean? Towards making meaning explicit in learning analytics design. Proceedings of the Australian Conference on Computer-Human Interaction (OzCHI ’17), 28 November–1 December 2017, Brisbane, Australia (pp. 528–532). New York: ACM. https://doi.org/10.1145/3152771.3156171

Goggins, S., Jahnke, I., & Wulf, V. (2012). CSCL@work revisited — Beyond CSCL and CSCW? Are there key design principles for computer supported collaborative learning at the workplace? Proceedings of the International Conference on Supporting Group Work (GROUP ’12), 27–31 October 2012, Sanibel Island, FL, USA (pp. 323–326). New York: ACM. https://doi.org/10.1145/2389176.2389239

Gomez, K., Kyza, E. A., & Mancevice, N. (2018). Participatory design and the learning sciences. In F. Fischer, C. E. Hmelo-Silver, S. R. Goldman, & P. Reimann (Eds.), International Handbook of the Learning Sciences (pp. 401–409). New York: Taylor and Francis. https://doi.org/10.4324/9781315617572-39

Goodyear, P. M., Banks, S., Hodgson, V., & McConnell, D. (2006). Advances in Research on Networked Learning (Vol. 4). Dordrecht, Netherlands: Springer Science & Business Media. https://doi.org/10.1007/1-4020-7909-5

Graesser, A., Dowell, N., Hampton, A. J., Lippert, A. M., Li, H., & Shaffer, D. W. (2018). Building intelligent conversational tutors and mentors for team collaborative problem solving: Guidance from the 2015 Program for International Student Assessment. In Building Intelligent Tutoring Systems for Teams (Vol. 19, pp. 173–211). Emerald Publishing Limited. https://doi.org/10.1108/S1534-085620180000019012

Greller, W., & Drachsler, H. (2012). Translating learning into numbers: A generic framework for learning analytics. Educational Technology & Society, 15(3), 42–57. https://www.jstor.org/stable/10.2307/jeductechsoci.15.3.42

Grudin, J. (1991). CSCW. Communications of the ACM, 34(12), 30–34. https://doi.org/10.1145/125319.125320

Haataja, E., Malmberg, J., & Järvelä, S. (2018). Monitoring in collaborative learning: Co-occurrence of observed behavior and physiological synchrony explored. Computers in Human Behavior, 87, 337–347. https://doi.org/10.1016/j.chb.2018.06.007

Harrer, A. (2013). Analytics of collaborative planning in Metafora: Architecture, data, and analytic methods. Proceedings of the Third International Conference on Learning Analytics & Knowledge (LAK ’13), 8–12 April 2013, Leuven, Belgium (pp. 255–259). New York: ACM. https://doi.org/10.1145/2460296.2460348

Herder, T., Swiecki, Z., Fougt, S. S., Tamborg, A. L., Allsopp, B. B., Shaffer, D. W., & Misfeldt, M. (2018). Supporting teachers’ intervention in students’ virtual collaboration using a network based model. Proceedings of the Eighth International Conference on Learning Analytics & Knowledge (LAK ’18), 5–9 March 2018, Sydney, Australia (pp. 21–25). New York: ACM. https://doi.org/10.1145/3170358.3170394

Hernandez-Leo, D., Martinez-Maldonado, R., Pardo, A., Muñoz-Cristobal, J., & Rodriguez-Triana, M. (2016). Analytics for learning design: A layered framework and tools. British Journal of Educational Technology, 50(1), 139–152. https://doi.org/10.1111/bjet.12645

Hernandez-Leo, D., Villasclaras-Fernandez, E. D., Asensio-Perez, J. I., Dimitriadis, Y. A., & Retalis, S. (2006). CSCL scripting patterns: Hierarchical relationships and applicability. Proceedings of the International Conference on Advanced Learning Technologies (ICALT ’06), 5–7 July 2006, Kerkrade, Netherlands (pp. 388–392). IEEE. https://doi.org/10.1109/ICALT.2006.1652452

Hernández Leo, D., Asensio-Pérez, J. I., Dimitriadis, Y., & Villasclaras-Fernández, E. D. (2010). Generating CSCL scripts: From a conceptual model of pattern languages to the design of real scripts. In P. Goodyear & S. Retalis (Eds.), Technology-Enhanced Learning: Design Patterns and Pattern Languages (pp. 49–64). Rotterdam: Sense Publishers. https://doi.org/10.1163/9789460910623_004

Hesse, F., Care, E., Buder, J., Sassenberg, K., & Griffin, P. (2015). A framework for teachable collaborative problem solving skills. In P. Griffin & E. Care (Eds.), Assessment and Teaching of 21st Century Skills: Methods and Approach (pp. 37–56). Dordrecht, Netherlands: Springer. https://doi.org/10.1007/978-94-017-9395-7_2

Hey, A., Tansley, S., & Tolle, K. (2009). The Fourth Paradigm: Data-Intensive Scientific Discovery. Redmond, WA, USA: Microsoft Research.

Isotani, S. (2011). Guest editorial: Special issue on intelligent and innovative support systems for CSCL. IEEE Transactions on Learning Technologies, 4(1), 1–4. https://doi.org/10.1109/TLT.2011.7

Järvelä, S., Malmberg, J., Haataja, E., Sobocinski, M., & Kirschner, P. A. (2019). What multimodal data can tell us about the students’ regulation of their learning process. Learning and Instruction, 4, in press. https://doi.org/10.1016/j.learninstruc.2019.04.004

Jeong, H., & Hmelo-Silver, C. E. (2010). Technology use in CSCL: A content meta-analysis. Proceedings of the Hawaii International Conference on System Sciences, 5–8 January 2010, Honolulu, HI, USA (pp. 1–10). IEEE. https://doi.org/10.1109/HICSS.2010.364

Jeong, H., Hmelo-Silver, C. E., & Jo, K. (2019). Ten years of computer-supported collaborative learning: A meta-analysis of CSCL in STEM education during 2005–2014. Educational Research Review, 28, 100284. https://doi.org/10.1016/j.edurev.2019.100284

Jermann, P., & Dillenbourg, P. (2008). Group mirrors to support interaction regulation in collaborative problem solving. Computers & Education, 51(1), 279–296. https://doi.org/10.1016/j.compedu.2007.05.012

Johnson, S. D., Suriya, C., Yoon, S. W., Berrett, J. V., & La Fleur, J. (2002). Team development and group processes of virtual learning teams. Computers & Education, 39, 379–393. https://doi.org/10.1016/S0360-1315(02)00074-X

Joksimovic, S., Gasevic, D., Kovanovic, V., Adesope, O., & Hatala, M. (2014). Psychological characteristics in cognitive presence of communities of inquiry: A linguistic analysis of online discussions. The Internet and Higher Education, 22, 1–10. https://doi.org/10.1016/j.iheduc.2014.03.001

Jørnø, R. L., & Gynther, K. (2018). What constitutes an “actionable insight” in learning analytics? Journal of Learning Analytics, 5(3), 198–221. https://doi.org/10.18608/jla.2018.53.13

Kasepalu, R., Santos, L. P. P., & Ley, T. T. (2019). Providing teachers with individual and group-level collaboration analytics: A paper prototype. Proceedings of the International Workshop on Collaboration Analytics: Making Learning Visible in Collaborative Settings (pp. 1–4). Held at the International Conference on Computer-Supported Collaborative Learning (CSCL ’19), 17–21 June 2019, Lyon, France. https://collaborationanalytics.files.wordpress.com/2019/06/submission-2-reet.pdf

Kay, J., Maisonneuve, N., Yacef, K., & Zaïane, O. (2006). Mining patterns of events in students’ teamwork data. Proceedings of the Workshop on Educational Data Mining (pp. 45–52). Held at the Eighth International Conference on Intelligent Tutoring Systems (ITS ’06), 26–30 June 2006, Jhongli, Taiwan. IEDMS. https://www.educationaldatamining.org/ITS2006EDM/Kay_Yacef.pdf

Khan, S. M. (2017). Multimodal behavioral analytics in intelligent learning and assessment systems. In A. A. von Davier, M. Zhu, & P. C. Kyllonen (Eds.), Innovative Assessment of Collaboration (pp. 173–184). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-33261-1_11

Kim, J., & Kumar, R. (Eds.). (2012). Proceedings of the Full-Day Workshop on Intelligent Support for Learning In Groups. Held at the 11th International Conference on Intelligent Tutoring Systems (ITS 2012), 15 June 2012, Chania, Greece. https://sites.google.com/site/islg2012/

Kim, J. W., Sottilare, R. A., Brawner, K., & Flowers, T. (2018). Integrating sensors and exploiting sensor data with GIFT for improved learning analytics. Proceedings of the Annual GIFT Users Symposium (GIFTSym6), 9–11 May 2018, Orlando, FL, USA (pp. 299–312). https://gifttutoring.org/attachments/download/2695/13_GIFTSym6_Exp_Analysis_Eval_paper_15.pdf

Kitto, K., Buckingham Shum, S., & Gibson, A. (2018). Embracing imperfection in learning analytics. Proceedings of the Eighth International Conference on Learning Analytics and Knowledge (LAK ’18), 5–9 March 2018, Sydney, Australia (pp. 451–460). New York: ACM. https://doi.org/10.1145/3170358.3170413

Knight, S., & Buckingham Shum, S. (2017). Theory and learning analytics. In C. Lang, G. Siemens, A. Wise, & D. Gasevic (Eds.), Handbook of Learning Analytics (pp. 17–22). Edmonton, AB, Canada: SOLAR. https://doi.org/10.18608/hla17.001

Knipfer, K., Prilla, M., Cress, U., & Herrmann, T. (2011). Computer support for collaborative reflection on captured teamwork data. Proceedings of the International Conference on Computer Supported Collaborative Learning (CSCL ’11), 4–8 July 2011, Hong Kong, China (pp. 938–939). ISLS.

Kollar, I., Fischer, F., & Hesse, F. W. (2006). Collaboration scripts — A conceptual analysis. Educational Psychology Review, 18(2), 159–185. https://doi.org/10.1007/s10648-006-9007-2

Kovanović, V., Joksimović, S., Waters, Z., Gašević, D., Kitto, K., Hatala, M., & Siemens, G. (2016). Towards automated content analysis of discussion transcripts: A cognitive presence case. Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK ’16), 25–29 April 2016, Edinburgh, Scotland (pp. 15–24). New York: ACM. https://doi.org/10.1145/2883851.2883950

Kreijns, K., Kirschner, P. A., & Jochems, W. (2002). The sociability of computer-supported collaborative learning environments. Journal of Educational Technology & Society, 5(1), 8–22. https://www.jstor.org/stable/jeductechsoci.5.1.8

Kumar, R., & Kim, J. (2014). Special issue on intelligent support for learning in groups. International Journal of Artificial Intelligence in Education, 24(1), 1–7. https://doi.org/10.1007/s40593-013-0013-5

Kumar, R., Rosé, C. P., Wang, Y.-C., Joshi, M., & Robinson, A. (2007). Tutorial dialogue as adaptive collaborative learning support. Proceedings of the International Conference on Artificial Intelligence in Education (AIED ’07), 9–13 July 2007, Los Angeles, CA, USA (pp. 383–390). IOS Press. https://dl.acm.org/doi/10.5555/1563601.1563663

Landauer, T. K., McNamara, D. S., Dennis, S., & Kintsch, W. (2013). Handbook of Latent Semantic Analysis. New York: Psychology Press.

Lee, A. V. Y., & Tan, S. C. (2017). Promising ideas for collective advancement of communal knowledge using temporal analytics and cluster analysis. Journal of Learning Analytics, 4(3), 76–101. https://doi.org/10.18608/jla.2017.43.5

Lim, L., Dawson, S., Joksimovic, S., & Gašević, D. (2019). Exploring students’ sensemaking of learning analytics dashboards: Does frame of reference make a difference? Proceedings of the Ninth International Conference on Learning Analytics & Knowledge (LAK ’19), 4–8 March 2019, Tempe, Arizona, USA (pp. 250–259). New York: ACM. https://doi.org/10.1145/3303772.3303804

Liu, A. L., & Nesbit, J. C. (2019). Dashboards for computer-supported collaborative learning. In M. Virvou, E. Alepis, G. A. Tsihrintzis, & L. C. Jain (Eds.), Machine Learning Paradigms: Advances in Learning Analytics (pp. 157–182). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-13743-4_9

Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing pedagogical action: Aligning learning analytics with learning design. American Behavioral Scientist, 57(10), 1439–1459. https://doi.org/10.1177/0002764213479367

Ludvigsen, S., Cress, U., Law, N., Stahl, G., & Rosé, C. P. (2017). Future direction for the CSCL field: Methodologies and eight controversies. International Journal of Computer-Supported Collaborative Learning, 12(4), 337–341. https://doi.org/10.1007/s11412-017-9268-4

Macfadyen, L. P., & Dawson, S. (2012). Numbers are not enough. Why e-learning analytics failed to inform an institutional strategic plan. Journal of Educational Technology & Society, 15(3), 149–163. https://www.jstor.org/stable/10.2307/jeductechsoci.15.3.149

Malmberg, J., Järvelä, S., Holappa, J., Haataja, E., Huang, X., & Siipo, A. (2018). Going beyond what is visible: What multichannel data can reveal about interaction in the context of collaborative learning? Computers in Human Behavior. https://doi.org/10.1016/j.chb.2018.06.030

Maltese, A. V., Harsh, J. A., & Svetina, D. (2015). Data visualization literacy: Investigating data interpretation along the novice–expert continuum. Journal of College Science Teaching, 45(1), 84–90. https://www.jstor.org/stable/43631889

Mandinach, E. B., & Gummer, E. S. (2013). A systemic view of implementing data literacy in educator preparation. Educational Researcher, 42(1), 30–37. https://doi.org/10.3102/0013189x12459803

Mangaroska, K., & Giannakos, M. N. (2018). Learning analytics for learning design: A systematic literature review of analytics-driven design to enhance learning. IEEE Transactions on Learning Technologies, 12(4), 516–534. https://doi.org/10.1109/TLT.2018.2868673

Martinez-Maldonado, R. (2019). A handheld classroom dashboard: Teachers’ perspectives on the use of real-time collaborative learning analytics. International Journal of Computer-Supported Collaborative Learning, 14(3), 383–411. https://doi.org/10.1007/s11412-019-09308-z

Martinez-Maldonado, R., Elliot, D., Axisa, C., Power, T., & Buckingham Shum, S. (2019a). Making the design of CSCL analytics interfaces a co-design process: The case of multimodal teamwork in healthcare. Proceedings of the International Conference on Computer-Supported Collaborative Work (CSCL ’19), 17–21 June 2019, Lyon, France (pp. 859–860). ISLS. https://repository.isls.org/handle/1/1698

Martinez-Maldonado, R., Fernandez, G., Echeverria, V., & Buckingham Shum, S. (2020). From data to insights: A layered storytelling approach for multimodal learning analytics. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’20), 25–30 April 2020, Honolulu, HI, USA (pp. 1–15). New York: ACM. https://doi.org/10.1145/3313831.3376148

Martinez-Maldonado, R., Goodyear, P., Kay, J., Thompson, K., & Carvalho, L. (2016). An actionable approach to understand group experience in complex, multi-surface spaces. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’16), 7–12 May 2016, San Jose, California, USA (pp. 2062–2074). New York: ACM. https://doi.org/10.1145/2858036.2858213

Martinez-Maldonado, R., Kay, J., Buckingham Shum, S., & Yacef, K. (2019b). Collocated collaboration analytics: Principles and dilemmas for mining multimodal interaction data. Human–Computer Interaction, 34(1), 1–50. https://doi.org/10.1080/07370024.2017.1338956

Martinez-Maldonado, R., Kay, J., Wallace, J., & Yacef, K. (2011). Modelling symmetry of activity as an indicator of collocated group collaboration. Proceedings of the International Conference on User Modeling, Adaptation and Personalization (UMAP ’11), 11–15 July 2011, Girona, Spain (pp. 196–204). Berlin: Springer. https://doi.org/10.1007/978-3-642-22362-4_18

Martinez-Maldonado, R., Buckingham Shum, S., Schneider, B., Charleer, S., Klerkx, J., & Duval, E. (2017). Learning analytics for natural user interfaces: A framework, case studies and a maturity analysis. Journal of Learning Analytics, 4(1), 24–57. https://doi.org/10.18608/jla.2017.41.4

Martinez-Maldonado, R., Worsley, M., Schneider, B., & Kharrufa, A. (2019c). International Workshop on Collaboration Analytics: Making learning visible in collaborative settings. Proceedings of the International Conference on Computer-Supported Collaborative Learning (CSCL ’19), 17–21 June 2019, Lyon, France (pp. 1044–1049). ISLS.

Matcha, W., Gasevic, D., & Pardo, A. (2019). A systematic review of empirical studies on learning analytics dashboards: A self-regulated learning perspective. IEEE Transactions on Learning Technologies, 13(2), 226–245. https://doi.org/10.1109/TLT.2019.2916802

Mazzocchi, F. (2015). Could Big Data be the end of theory in science? A few remarks on the epistemology of data‐driven science. EMBO Reports, 16(10), 1250–1255. https://doi.org/10.15252/embr.201541001

McNamara, D. S., Louwerse, M. M., McCarthy, P. M., & Graesser, A. C. (2010). Coh-Metrix: Capturing linguistic features of cohesion. Discourse Processes, 47(4), 292–330. https://doi.org/10.1080/01638530902959943

Milligan, S. K. (2020). Standards for developing assessments of learning using process data. In M. Bearman, D. Boud, P. Dawson, J. Tai, & R. Ajjawi (Eds.), Re-imagining University Assessment in a Digital World (pp. 179–192). Dordrecht, Netherlands: Springer. https://doi.org/10.1007/978-3-030-41956-1_13

Milligan, S. K., & Griffin, P. (2016). Understanding learning and learning design in MOOCs: A measurement-based interpretation. Journal of Learning Analytics, 3(2), 88–115. https://doi.org/10.18608/jla.2016.32.5

Mislevy, R. J. (2019). Advances in measurement and cognition. The Annals of the American Academy of Political and Social Science, 683(1), 164–182. https://doi.org/10.1177%2F0002716219843816

Mor, Y., Ferguson, R., & Wasson, B. (2015). Editorial: Learning design, teacher inquiry into student learning and learning analytics: A call for action. British Journal of Educational Technology, 46(2), 221–229. https://doi.org/10.1111/bjet.12273

Noroozi, O., Alikhani, I., Järvelä, S., Kirschner, P. A., Juuso, I., & Seppänen, T. (2019). Multimodal data to design visual learning analytics for understanding regulation of learning. Computers in Human Behavior, 100(Nov 2019), 298–304. https://doi.org/10.1016/j.chb.2018.12.019

Olsen, J. (2017). Orchestrating Combined Collaborative and Individual Learning in the Classroom (doctoral dissertation). Carnegie Mellon University, Pittsburgh, PA, USA.

Pardo, A., Mirriahi, N., Martinez-Maldonado, R., Jovanovic, J., Dawson, S., & Gašević, D. (2016). Generating actionable predictive models of academic performance. Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK ’16), 25–29 April 2016, Edinburgh, Scotland (pp. 474–478). New York: ACM. https://doi.org/10.1145/2883851.2883870

Pauleen, D. (2004). Virtual Teams: Projects, Protocols and Processes. Hershey, PA, USA: IGI Global.

Perera, D., Kay, J., Koprinska, I., Yacef, K., & Zaïane, O. R. (2008). Clustering and sequential pattern mining of online collaborative learning data. IEEE Transactions on Knowledge and Data Engineering, 21(6), 759–772. https://doi.org/10.1109/TKDE.2008.138

Persico, D., & Pozzi, F. (2015). Informing learning design with learning analytics to improve teacher inquiry. British Journal of Educational Technology, 46(2), 230–248. https://doi.org/10.1111/bjet.12207

Pijeira-Díaz, H. J., Drachsler, H., Järvelä, S., & Kirschner, P. A. (2016). Investigating collaborative learning success with physiological coupling indices based on electrodermal activity. Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK ’16), 25–29 April 2016, Edinburgh, Scotland (pp. 64–73). New York: ACM. https://doi.org/10.1145/2883851.2883897

Porter, G., & Beyerlein, M. (2000). Historic roots of team theory and practice. In M. M. Beyerlein (Ed.), Work Teams: Past, Present and Future (pp. 3–24). Dordrecht, Netherlands: Springer. https://doi.org/10.1007/978-94-015-9492-9_1

Praharaj, S., Scheffel, M., Drachsler, H., & Specht, M. (2018). Multimodal analytics for real-time feedback in co-located collaboration. Proceedings of the European Conference on Technology Enhanced Learning (ECTEL ’18), 3–6 September 2018, Leeds, UK (pp. 187–201). Springer. https://doi.org/10.1007/978-3-319-98572-5_15

Prieto-Alvarez, C., Martinez-Maldonado, R., & Buckingham Shum, S. (2018). Mapping learner-data journeys: Evolution of a visual co-design tool. Proceedings of the ACM Australian Computer-Human Interaction Conference (OzCHI ’18), 5–7 December 2018, Melbourne, Australia (pp. 205–214). New York: ACM. https://doi.org/10.1145/3292147.3292168

Prieto-Alvarez, C., Martinez-Maldonado, R., & Buckingham Shum, S. (2020). LA-DECK: A card-based learning analytics co-design tool. Proceedings of the 10th International Learning Analytics and Knowledge Conference (LAK ’20), 23–27 March 2020, Frankfurt, Germany (pp. 63–72). New York: ACM. https://doi.org/10.1145/3375462.3375476

Prieto, L. P., Rodríguez Triana, M. J., Martínez Maldonado, R., Dimitriadis, Y. A., & Gašević, D. (2018). Orchestrating learning analytics (OrLA): Supporting inter-stakeholder communication about adoption of learning analytics at the classroom level. Australasian Journal of Educational Technology, 35(4), 14–33. https://doi.org/10.14742/ajet.4314

Prinsloo, P., & Slade, S. (2016). Student vulnerability, agency and learning analytics: An exploration. Journal of Learning Analytics, 3(1), 159–182. https://doi.org/10.18608/jla.2016.31.10

Puntambekar, S., Erkens, G., & Hmelo-Silver, C. (2011). Analyzing Interactions in CSCL: Methods, Approaches and Issues (Vol. 12). Boston, MA, USA: Springer Science & Business Media. https://doi.org/10.1007/978-1-4419-7710-6

Reimann, P. (2016). Connecting learning analytics with learning research: The role of design-based research. Learning: Research and Practice, 2(2), 130–142. https://doi.org/10.1080/23735082.2016.1210198

Reimann, P., Yacef, K., & Kay, J. (2011). Analyzing collaborative interactions with data mining methods for the benefit of learning. In S. Puntambekar, G. Erkens, & C. Hmelo-Silver (Eds.), Analyzing Interactions in CSCL: Methods, Approaches and Issues (pp. 161–185). Boston, MA, USA: Springer. https://doi.org/10.1007/978-1-4419-7710-6_8

Rodríguez Triana, M. J., Martínez Monés, A., Asensio Pérez, J. I., & Dimitriadis, Y. (2014). Scripting and monitoring meet each other: Aligning learning analytics and learning design to support teachers in orchestrating CSCL situations. British Journal of Educational Technology, 46(2), 330–343. https://doi.org/10.1111/bjet.12198

Rodríguez-Triana, M. J., Prieto, L. P., Martínez-Monés, A., Asensio-Pérez, J. I., & Dimitriadis, Y. (2018a). Monitoring Collaborative Learning Activities: Exploring the Differential Value of Collaborative Flow Patterns for Learning Analytics. Proceedings of the 18th International Conference on Advanced Learning Technologies (ICALT ’18), 9–13 July 2018, Mumbai, India (pp. 155–159). IEEE. https://doi.org/10.1109/ICALT.2018.00044

Rodriguez Triana, M. J., Prieto, L. P., Martínez-Monés, A., Asensio Pérez, J. I., & Dimitriadis, Y. (2018b). The teacher in the loop: Customizing MMLA in blended CSCL scenarios. Proceedings of the Eighth International Conference on Learning Analytics and Knowledge (LAK ’18), 5–9 March 2018, Sydney, Australia (pp. 417–426). New York: ACM. https://doi.org/10.1145/3170358.3170364

Rogers, T., Gašević, D., & Dawson, S. (2016). Learning analytics and the imperative for theory driven research. In C. Haythornthwaite, R. Andrews, J. Fransman, & E. M. Meyers (Eds.), The SAGE Handbook of E-Learning Research (pp. 232–250). Thousand Oaks, CA, USA: SAGE Publications Ltd. http://doi.org/10.4135/9781473955011.n12

Rolim, V., Ferreira, R., Lins, R. D., & Gǎsević, D. (2019). A network-based analytic approach to uncovering the relationship between social and cognitive presences in communities of inquiry. The Internet and Higher Education, 42, 53–65. https://doi.org/10.1016/j.iheduc.2019.05.001

Rosé, C. (2017). Discourse analytics. In C. Lang, G. Siemens, A. Wise, & D. Gasevic (Eds.), Handbook of Learning Analytics (pp. 105–114). Edmonton, AB, Canada: SOLAR. https://doi.org/10.18608/hla17.009

Rosé, C., Wang, Y.-C., Cui, Y., Arguello, J., Stegmann, K., Weinberger, A., & Fischer, F. (2008). Analyzing collaborative learning processes automatically: Exploiting the advances of computational linguistics in computer-supported collaborative learning. International Journal of Computer-Supported Collaborative Learning, 3(3), 237–271. https://doi.org/10.1007/s11412-007-9034-0

Rosé, C. P. (2018). Learning analytics in the learning sciences. In F. Fischer, C. E. Hmelo-Silver, S. R. Goldman, & P. Reimann (Eds.), International Handbook of the Learning Sciences (pp. 511–519). New York: Routledge.

Rummel, N., Walker, E., & Aleven, V. (2016). Different futures of adaptive collaborative learning support. International Journal of Artificial Intelligence in Education, 26(2), 784–795. https://doi.org/10.1007/s40593-016-0102-3

Rummel, N., Weinberger, A., Wecker, C., Fischer, F., Meier, A., Voyiatzaki, E., . . . Joshi, M. (2008). New challenges in CSCL: Towards adaptive script support. Proceedings of the International Conference for the Learning Sciences (ICLS ’08), 23–28 June 2008, Utrecht, Netherlands (pp. 338–345). ISLS. https://dl.acm.org/doi/10.5555/1599936.1600036

Rupp, A. A., Nugent, R., & Nelson, B. (Eds.). (2012). Special issue on diagnostic measurement in complex learning environments using evidence-centered design. Journal of Educational Data Mining, 4, 1–230. https://jedm.educationaldatamining.org/index.php/JEDM/issue/view/10

Salas, E., Sims, D. E., & Burke, S. (2005). Is there a “Big Five” in teamwork? Small Group Research, 36(5), 1–46. https://doi.org/10.1177/1046496405277134

Sanders, E. B.-N., & Stappers, P. J. (2008). Co-creation and the new landscapes of design. Co-design, 4(1), 5–18. https://doi.org/10.1080/15710880701875068

Scardamalia, M., & Bereiter, C. (2006). Knowledge building: Theory, pedagogy, and technology. In R. K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences (pp. 97–115). New York: Cambridge University Press. https://psycnet.apa.org/record/2006-07157-007

Schneider, B. (2020). A methodology for capturing joint visual attention using mobile eye-trackers. Journal of Visualized Experiments (155), e60670. https://doi.org/10.3791/60670

Schneider, B., Dich, Y., & Radu, I. (2020). Unpacking the relationship between existing and new measures of physiological synchrony and collaborative learning: A mixed methods study. International Journal of Computer-Supported Collaborative Learning, 15, 89–113. https://doi.org/10.1007/s11412-020-09318-2

Schneider, B., & Pea, R. (2017). Real-time mutual gaze perception enhances collaborative learning and collaboration quality. Educational Media and Technology Yearbook, 40, 99–125. https://doi.org/10.1007/s11412-013-9181-4

Schneider, B., Worsley, M., & Martinez-Maldonado, R. (2021). Gesture and gaze: Multimodal data in dyadic interactions. In U. Cress, C. Rosé, A. Wise, & J. Oshima (Eds.), International Handbook of Computer-Supported Collaborative Learning (in press). London, UK: Springer.

Sergis, S., & Sampson, D. G. (2017). Teaching and learning analytics to support teacher inquiry: A systematic literature review. In A. Peña-Ayala (Ed.), Learning Analytics: Fundaments, Applications, and Trends (pp. 25–63). Basel, Switzerland: Springer. https://doi.org/10.1007/978-3-319-52977-6_2

Shaffer, D. W. (2017). Quantitative ethnography. Madison, WI, USA: Cathcart Press.

Shaffer, D. W., Collier, W., & Ruis, A. R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3(3), 9–45. https://doi.org/10.18608/jla.2016.33.3

Shute, V. J., & Ventura, M. (2013). Stealth Assessment: Measuring and Supporting Learning in Video Games. Cambridge, MA, USA: MIT Press. https://mitpress.mit.edu/books/stealth-assessment

Siemens, G., & Long, P. (2011). Penetrating the fog: Analytics in learning and education. Educause Review, 46(5), 30–32. https://doi.org/10.17471/2499-4324/195

Soller, A., Martinez, A., Jermann, P., & Muehlenbrock, M. (2005). From mirroring to guiding: A review of state of the art technology for supporting collaborative learning. International Journal of Artificial Intelligence in Education, 15(4), 261–290. https://dl.acm.org/doi/10.5555/1434935.1434937

Spikol, D., Ruffaldi, E., & Cukurova, M. (2017a). Using multimodal learning analytics to identify aspects of collaboration in project-based learning. Proceedings of the International Conference on Computer Supported Learning (CSCL ’17), 18–22 June 2017, Philadelphia, PA, USA (pp. 263–270). ISLS.

Spikol, D., Ruffaldi, E., Landolfi, L., & Cukurova, M. (2017b). Estimation of success in collaborative learning based on multimodal learning analytics features. Proceedings of the International Conference on Advanced Learning Technologies (ICALT ’17), 3–7 July 2017, Timisoara, Romania (pp. 269–273). https://doi.org/10.1109/ICALT.2017.122

Stadler, M., Herborn, K., Mustafić, M., & Greiff, S. (2020). The assessment of collaborative problem solving in PISA 2015: An investigation of the validity of the PISA 2015 CPS tasks. Computers & Education, 157, 103964. https://doi.org/10.1016/j.compedu.2020.103964

Stahl, G. (2004). Building collaborative knowing: Elements of a social theory of CSCL. In J.-W. Strijbos, P. A. Kirschner, & R. L. Martens (Eds.), What We Know about CSCL: And Implementing It in Higher Education (pp. 53–85). Dordrecht, Netherlands: Springer. https://doi.org/10.1007/1-4020-7921-4_3

Stahl, G. (2006). Group Cognition: Computer Support for Building Collaborative Knowledge. Cambridge, MA, USA: MIT Press. https://mitpress.mit.edu/books/group-cognition

Stahl, G. (2013). Theories of collaborative cognition: Foundations for CSCL and CSCW together. In S. P. Goggins, I. Jahnke, & V. Wulf (Eds.), Computer-Supported Collaborative Learning at the Workplace: CSCL@Work (pp. 43–63). Boston, MA, USA: Springer. https://doi.org/10.1007/978-1-4614-1740-8_3

Stahl, G. (2017). Global Introduction to CSCL. Philadelphia, PA, USA: Lulu.

Stahl, G., & Hakkarainen, K. (2021). Theories of CSCL. In U. Cress, C. Rosé, A. Wise, & J. Oshima (Eds.), International Handbook of Computer Supported Collaborative Learning (in press). London, UK: Springer.

Steier, R., Shapiro, B., Christidou, D., Pierroux, P., Davidsen, J., & Hall, R. (2019). Tools and methods for “4E analysis”: New lenses for analyzing interaction in CSCL. Proceedings of the International Conference on Computer Supported Collaborative Learning (CSCL ’19), 17–21 June 2019, Lyon, France (pp. 759–766). ISLS.

Stevens, R., Galloway, T., & Willemsen-Dunlap, A. (2020). Approaches for inserting neurodynamics into the training of healthcare teams. In C. S. Nam (Ed.), Neuroergonomics: Principles and Practice (pp. 251–269). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-34784-0_13

Suthers, D. D. (2017). Multilevel analysis of activity and actors in heterogeneous networked learning environments. In C. Lang, G. Siemens, A. Wise, & D. Gasevic (Eds.), Handbook of Learning Analytics (pp. 189–197). Edmonton, AB, Canada: SOLAR. https://doi.org/10.18608/hla17.016

Suthers, D. D., Dwyer, N., Medina, R., & Vatrapu, R. (2010). A framework for conceptualizing, representing, and analyzing distributed interaction. International Journal of Computer-Supported Collaborative Learning, 5(1), 5–42. https://doi.org/10.1007/s11412-009-9081-9

Swiecki, Z., Ruis, A., Farrell, C., & Shaffer, D. W. (2020). Assessing individual contributions to collaborative problem solving: A network analysis approach. Computers in Human Behavior, 104(March 2020), 105876. https://doi.org/10.1016/j.chb.2019.01.009

Szewkis, E., Nussbaum, M., Rosen, T., Abalos, J., Denardin, F., Caballero, D., . . . Alcoholado, C. (2011). Collaboration within large groups in the classroom. International Journal of Computer-Supported Collaborative Learning, 6(4), 561–575. https://doi.org/10.1007/s11412-011-9123-y

Tan, J. P.-L., Yang, S., Koh, E., & Jonathan, C. (2016). Fostering 21st century literacies through a collaborative critical reading and learning analytics environment: User-perceived benefits and problematics. Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (LAK ’16), 25–29 April 2016, Edinburgh, Scotland (pp. 430–434). New York: ACM. https://doi.org/10.1145/2883851.2883965

Tarmazdi, H., Vivian, R., Szabo, C., Falkner, K., & Falkner, N. (2015). Using learning analytics to visualise computer science teamwork. Proceedings of the Conference on Innovation and Technology in Computer Science Education (ITiCSE ’15), 6–8 July 2015, Vilnius, Lithuania (pp. 165–170). New York: ACM. https://doi.org/10.1145/2729094.2742613

Tausczik, Y. R., & Pennebaker, J. W. (2010). The psychological meaning of words: LIWC and computerized text analysis methods. Journal of Language and Social Psychology, 29(1), 24–54. https://doi.org/10.1177%2F0261927X09351676

Tchounikine, P. (2019). Learners’ agency and CSCL technologies: Towards an emancipatory perspective. International Journal of Computer-Supported Collaborative Learning, 14(2), 237–250. https://doi.org/10.1007/s11412-019-09302-5

Teasley, S. D. (1997). Talking about reasoning: How important is the peer in peer collaboration? In L. B. Resnick, R. Säljö, C. Pontecorvo, & B. Burge (Eds.), Discourse, tools and reasoning: Essays on situated cognition. Heidelberg, Germany: Springer-Verlag. https://doi.org/10.1007/978-3-662-03362-3_16

Tempelaar, D. T., Rienties, B., & Nguyen, Q. (2017). Towards actionable learning analytics using dispositions. IEEE Transactions on Learning Technologies, 10(1), 6–16. https://doi.org/10.1109/TLT.2017.2662679

Tuckman, B. W. (1965). Developmental sequence in small groups. Psychological Bulletin, 63(6), 384–399. https://psycnet.apa.org/doi/10.1037/h0022100

van Leeuwen, A. (2019). Teachers’ perceptions of the usability of learning analytics reports in a flipped university course: When and how does information become actionable knowledge? Educational Technology Research and Development, 67(5), 1043–1064. https://doi.org/10.1007/s11423-018-09639-y

van Leeuwen, A., Janssen, J., Erkens, G., & Brekelmans, M. (2014). Supporting teachers in guiding collaborating students: Effects of learning analytics in CSCL. Computers and Education, 79, 28–39. https://doi.org/10.1016/j.compedu.2014.07.007

van Leeuwen, A., Janssen, J., Erkens, G., & Brekelmans, M. (2015). Teacher regulation of multiple computer-supported collaborating groups. Computers in Human Behavior, 52, 233–242. http://doi.org/10.1016/j.chb.2015.05.058

van Leeuwen, A., & Rummel, N. (2019). Orchestration tools to support the teacher during student collaboration: A review. Unterrichtswissenschaft, 47(2), 143–158. https://doi.org/10.1007/s42010-019-00052-9

van Leeuwen, A., Rummel, N., & van Gog, T. (2019). What information should CSCL teacher dashboards provide to help teachers interpret CSCL situations? International Journal of Computer-Supported Collaborative Learning, 14, 261–289. https://doi.org/10.1007/s11412-019-09299-x

van Schuppen, J. H. (2015). What is team theory? In J. H. van Schuppen & T. Villa (Eds.), Coordination Control of Distributed Systems (pp. 147–154). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-10407-2_18

Vrzakova, H., Amon, M. J., Stewart, A., Duran, N. D., & D’Mello, S. K. (2020). Focused or stuck together: multimodal patterns reveal triads’ performance in collaborative problem solving. Proceedings of the 10th International Conference on Learning Analytics & Knowledge (LAK ’20), 23–27 March 2020, Frankfurt, Germany (pp. 295–304). New York: ACM. https://doi.org/10.1145/3375462.3375467

Vujovic, M., Tassani, S., & Hernández-Leo, D. (2019). Motion capture as an instrument in multimodal collaborative learning analytics. Proceedings of the European Conference on Technology Enhanced Learning (ECTEL ’19), 16–19 September 2019, Delft, Netherlands (pp. 604–608). Springer. https://doi.org/10.1007/978-3-030-29736-7_49

Wake, J., Wasson, B., Bjørgen, E., & Heimsæter, F. (2019). Supporting firefighter training through visualising indoor positioning, motion and time use data. Proceedings of the WiPe Paper — Workshop on Advance Technologies for First Responders at International Conference on Information Systems for Crisis Response and Management (ISCRAM ’19), 19–22 May 2019, Valencia, Spain (pp. 87–90). ISCRAM. https://hdl.handle.net/11250/2650379

Wang, Q., Jin, H., & Liu, Y. (2010). Collaboration analytics: Mining work patterns from collaboration activities. Proceedings of the International Conference on Information and Knowledge Management (CIKM ’10), 26–30 October 2010, Toronto, ON, Canada (pp. 1861–1864). New York: ACM. https://doi.org/10.1145/1871437.1871748

West, D., Luzeckyj, A., Toohey, D., Vanderlelie, J., & Searle, B. (2020). Do academics and university administrators really know better? The ethics of positioning student perspectives in learning analytics. Australasian Journal of Educational Technology, 36(2), 60–70. https://doi.org/10.14742/ajet.4653

Wilder, C. R., & Ozgur, C. O. (2015). Business analytics curriculum for undergraduate majors. INFORMS Transactions on Education, 15(2), 180–187. https://doi.org/10.1287/ited.2014.0134

Wiley, K. J., Dimitriadis, Y., Bradford, A., & Linn, M. C. (2020). From theory to action: Developing and evaluating learning analytics for learning design. Proceedings of the 10th International Conference on Learning Analytics & Knowledge (LAK ’20), 23–27 March 2020, Frankfurt, Germany (pp. 569–578). New York: ACM. https://doi.org/10.1145/3375462.3375540

Winne, P. H. (2017). Leveraging big data to help each learner and accelerate learning science. Teachers College Record, 119(3), 1–24.

Wise, A., Azevedo, R., Stegmann, K., Malmberg, J., Rose, C. P., Mudrick, N., . . . Fischer, F. (2015). CSCL and learning analytics: Opportunities to support social interaction, self-regulation and socially shared regulation. Proceedings of the International Conference on Computer Supported Learning (CSCL ’15), 7–11 June 2015, Gothenburg, Sweden (pp. 607–614). ISLS. https://repository.isls.org/handle/1/446

Wise, A., Knight, S., & Buckingham Shum, S. (2021). Collaborative learning analytics. In U. Cress, C. Rosé, A. Wise, & J. Oshima (Eds.), International Handbook of Computer-Supported Collaborative Learning (in press). London, UK: Springer.

Wise, A. F. (2014). Designing pedagogical interventions to support student use of learning analytics. Proceedings of the Fourth International Conference on Learning Analytics & Knowledge (LAK ’14), 24–28 March 2014, Indianapolis, USA (pp. 203–211). New York: ACM. https://doi.org/10.1145/2567574.2567588

Wise, A. F., & Schwarz, B. B. (2017). Visions of CSCL: Eight provocations for the future of the field. International Journal of Computer-Supported Collaborative Learning, 12(4), 423–467. https://doi.org/10.1007/s11412-017-9267-5

Wise, A. F., & Shaffer, D. W. (2015). Why theory matters more than ever in the age of big data. Journal of Learning Analytics, 2(2), 5–13. https://doi.org/10.18608/jla.2015.22.2

Worsley, M. (2019). Computationally augmented ethnography: Emotion tracking and learning in museum games. Proceedings of the Advances in Quantitative Ethnography: First International Conference (ICQE ’19), 19–22 October 2019, Madison, WI, USA (pp. 141–153). Springer Nature. https://doi.org/10.1007/978-3-030-33232-7_12

Worsley, M., Abrahamson, D., Blikstein, P., Grover, S., Schneider, B., & Tissenbaum, M. (2016). Situating multimodal learning analytics. Proceedings of the International Conference of the Learning Sciences (ICLS ’16), 20–24 June 2016, Singapore (pp. 1346–1349). ISLS.

Worsley, M., & Blikstein, P. (2018). A multimodal analysis of making. International Journal of Artificial Intelligence in Education, 28(3), 385–419. https://doi.org/10.1007/s40593-017-0160-1

Wu, L., Waber, B., Aral, S., Brynjolfsson, E., & Pentland, S. (2008). Mining face-to-face interaction networks using sociometric badges: Predicting productivity in an IT configuration task. Proceedings of the International Conference on Information Systems (ICIS ’08), 19–20 July 2008, Paris, France (pp. 1–19). AIS/ICIS. https://aisel.aisnet.org/icis2008/127

Xing, W., Wadholm, B., & Goggins, S. (2014). Learning analytics in CSCL with a focus on assessment: An exploratory study of activity theory-informed cluster analysis. Proceedings of the Fourth International Conference on Learning Analytics and Knowledge (LAK ’14), 24–28 March 2014, Indianapolis, USA (pp. 59–67). New York: ACM. https://doi.org/10.1145/2567574.2567587

What Do You Mean by Collaboration Analytics?

A Conceptual Model

Authors

DOI:

Keywords:

Abstract

References

Downloads

Published

How to Cite

Issue

Section

License

Most read articles by the same author(s)