Visual Learning Analytics for Educational Interventions in Primary and Secondary Schools
A Scoping Review
DOI:
https://doi.org/10.18608/jla.2024.8309Keywords:
visual learning analytics, learning analytics dashboard, educational interventions, primary school, secondary school, scoping review, systematic review, research paperAbstract
Visual Learning Analytics (VLA) uses analytics to monitor and assess educational data by combining visual and automated analysis to provide educational explanations. Such tools could aid teachers in primary and secondary schools in making pedagogical decisions, however, the evidence of their effectiveness and benefits is still limited. With this scoping review, we provide a comprehensive overview of related research on proposed VLA methods, as well as identifying any gaps in the literature that could assist in describing new and helpful directions to the field. This review searched all relevant articles in five accessible databases — Scopus, Web of Science, ERIC, ACM, and IEEE Xplore — using 40 keywords. These studies were mapped, categorized, and summarized based on their objectives, the collected data, the intervention approaches employed, and the results obtained. The results determined what affordances the VLA tools allowed, what kind of visualizations were used to inform teachers and students, and, more importantly, positive evidence of educational interventions. We conclude that there are moderate-to-clear learning improvements within the limit of the studies’ interventions to support the use of VLA tools. More systematic research is needed to determine whether any learning gains are translated into long-term improvements.
References
Aguerrebere, C., He, H., Kwet, M., Laakso, M.-J., Lang, C., Marconi, C., Price-Dennis, D., & Zhang, H. (2022). Global perspectives on learning analytics in K12 education. In C. Lang, G. Siemens, A. F. Wise, D. Gašević, & A. Merceron (Eds.), Handbook of learning analytics (2nd ed) (pp. 223–231). Society for Learning Analytics Research. https://doi.org/10.18608/hla22
Alwahaby, H., Cukurova, M., Papamitsiou, Z., & Giannakos, M. (2022). The evidence of impact and ethical considerations of multimodal learning analytics: A systematic literature review. In M. Giannakos, D. Spikol, D. Di Mitri, K. Sharma, X. Ochoa, & R. Hammad (Eds.), The multimodal learning analytics handbook (pp. 289–325). Springer Cham. https://doi.org/10.1007/978-3-031-08076-0_12
Amoia, M., Brétaudière, T., Denis, A., Gardent, C., & Perez-Beltrachini, L. (2012). A serious game for second language acquisition in a virtual environment. Systemics, Cybernetics and Informatics, 10(1), 24–34. https://www.iiisci.org/Journal/pdv/sci/pdfs/HEA308SP.pdf
Apiola, M.-V., Lipponen, S., Seitamaa, A., Korhonen, T., & Hakkarainen, K. (2022). Learning analytics for knowledge creation and inventing in K–12: A systematic review. Intelligent Computing: Proceedings of the 2022 Computing Conference, 14–15 July 2022, Virtual (pp. 238–257). Springer, Cham. https://doi.org/10.1007/978-3-031-10467-1_15
Arksey, H., & O’Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19–32. https://doi.org/10.1080/1364557032000119616
Bai, Y., Xie, Y., Luo, W., & Yang, M. (2022). A study of strategies for data visualization to promote effective learning for primary school students. Blended Learning: Engaging Students in the New Normal Era. Proceedings of the 15th International Conference on Blended Learning (ICBL 2022), 19–22 December 2022, Hong Kong, China (pp. 108–120). IEEE Computer Society. https://doi.org/10.1007/978-3-031-08939-8
Camacho, V. L., de la Guía, E., Olivares, T., Flores, M. J., & Orozco-Barbosa, L. (2020). Data capture and multimodal learning analytics focused on engagement with a new wearable IoT approach. IEEE Transactions on Learning Technologies, 13(4), 704–717. https://doi.org/10.1109/tlt.2020.2999787
Chen, C.-M., & Chen, M.-C. (2009). Mobile formative assessment tool based on data mining techniques for supporting web-based learning. Computers & Education, 52(1), 256–273. https://doi.org/10.1016/j.compedu.2008.08.005
Chen, C.-M., Li, M.-C., & Liao, C.-K. (2023). Developing a collaborative writing system with visualization interaction network analysis to facilitate online learning performance. Interactive Learning Environments, 31(9), 6054–6073. https://doi.org/10.1080/10494820.2022.2028851
Chen, S., Ouyang, F., & Jiao, P. (2021). Promoting student engagement in online collaborative writing through a student‐facing social learning analytics tool. Journal of Computer Assisted Learning, 38(1), 192–208. https://doi.org/10.1111/jcal.12604
Delgado, A. J., Wardlow, L., McKnight, K., & O’Malley, K. (2015). Educational technology: A review of the integration, resources, and effectiveness of technology in K–12 classrooms. Journal of Information Technology Education: Research, 14, 397–416. http://www.jite.org/documents/Vol14/JITEv14ResearchP397-416Delgado1829.pdf
Dickler, R., Gobert, J., & Sao Pedro, M. (2021). Using innovative methods to explore the potential of an alerting dashboard for science inquiry. Journal of Learning Analytics, 8(2), 105–122. https://doi.org/10.18608/jla.2021.7153
Du, X., Yang, J., Shelton, B. E., Hung, J.-L., & Zhang, M. (2021). A systematic meta-review and analysis of learning analytics research. Behaviour & Information Technology, 40(1), 49–62. https://doi.org/10.1080/0144929x.2019.1669712
Ez-Zaouia, M., Tabard, A., & Lavoué, E. (2020). PROGDASH: Lessons learned from a learning dashboard in-the-wild. Proceedings of the 12th International Conference on Computer Supported Education (CSEDU 2020), 2–4 May 2020, Virtual (Vol. 2, pp. 105–117). ScitePress. https://doi.org/10.5220/0009424801050117
Farrell, T., Mikroyannidis, A., & Alani, H. (2017). “We’re seeking relevance”: Qualitative perspectives on the impact of learning analytics on teaching and learning. Proceedings of the 12th European Conference on Technology Enhanced Learning (EC-TEL 2017), 12–15 September 2017, Tallinn, Estonia (pp. 397–402). Springer. https://doi.org/10.1007/978-3-319-66610-5
Govaerts, S., Verbert, K., Duval, E., & Pardo, A. (2012). The student activity meter for awareness and self-reflection. CHI Conference on Human Factors in Computing Systems: Extended Abstracts (CHI EA ’12), 5–10 May 2012, Austin, TX, USA (pp. 869–884). ACM Press. https://doi.org/10.1145/2212776.2212860
Hantoobi, S., Wahdan, A., Al-Emran, M., & Shaalan, K. (2021). A review of learning analytics studies. In M. Al-Emran & K. Shaalan (Eds.), Recent advances in technology acceptance models and theories (pp. 119–134). Springer Cham. https://doi.org/10.1007/978-3-030-64987-6_8
Hew, K. F., Jia, C., Gonda, D. E., & Bai, S. (2020). Transitioning to the “new normal” of learning in unpredictable times: Pedagogical practices and learning performance in fully online flipped classrooms. International Journal of Educational Technology in Higher Education, 17(1), 57. https://doi.org/10.1186/s41239-020-00234-x
Hirsto, L., Saqr, M., López-Pernas, S., & Valtonen, T. (2022). A systematic narrative review of learning analytics research in K–12 and schools. Proceedings of the Finnish Learning Analytics and Artificial Intelligence in Education Conference (FLAIEC 22), 29–30 September 2022, Joensuu, Finland. CEUR Workshop Proceedings. https://ceur-ws.org/Vol-3383/FLAIEC22_paper_9536.pdf
Hu, L., Wu, J., & Chen, G. (2022). iTalk–iSee: A participatory visual learning analytical tool for productive peer talk. International Journal of Computer-Supported Collaborative Learning, 17(3), 397–425. https://doi.org/10.1007/s11412-022-09374-w
Kay, R., MacDonald, T., & DiGiuseppe, M. (2018). A comparison of lecture-based, active, and flipped classroom teaching approaches in higher education. Journal of Computing in Higher Education, 31(3), 449–471. https://doi.org/10.1007/s12528-018-9197-x
Kim, R., Olfman, L., Ryan, T., & Eryilmaz, E. (2014). Leveraging a personalized system to improve self-directed learning in online educational environments. Computers & Education, 70, 150–160. https://doi.org/10.1016/j.compedu.2013.08.006
Kimmons, R., Graham, C. R., & West, R. E. (2020). The PICRAT model for technology integration in teacher preparation. Contemporary Issues in Technology and Teacher Education Journal, 20(1). https://citejournal.org/volume-20/issue-1-20/general/the-picrat-model-for-technology-integration-in-teacher-preparation
Kippers, W. B., Poortman, C. L., Schildkamp, K., & Visscher, A. J. (2018). Data literacy: What do educators learn and struggle with during a data use intervention? Studies in Educational Evaluation, 56, 21–31. https://doi.org/10.1016/j.stueduc.2017.11.001
Koehler, M. J., & Mishra, P. (2009). What is technological pedagogical content knowledge (TPACK)? Contemporary Issues in Technology and Teacher Education, 9(1). https://citejournal.org/volume-9/issue-1-09/general/what-is-technological-pedagogicalcontent-knowledge
Kondo, T., Yokoyama, K., Misono, T., Inaba, R., & Watanabe, Y. (2021). Nudge for note taking assist system: A learning strategy feedback system among learners through their tablet. Proceedings of the 8th International Conference on Learning and Collaboration Technologies (LCT 2021), 24–29 July 2021, Virtual (pp. 315–331). Springer. https://doi.org/10.1007/978-3-030-77889-7_22
Larrabee Sønderlund, A., Hughes, E., & Smith, J. (2019). The efficacy of learning analytics interventions in higher education: A systematic review. British Journal of Educational Technology, 50(5), 2594–2618. https://doi.org/10.1111/bjet.12720
Lee, A. V. Y. (2021). Determining quality and distribution of ideas in online classroom talk using learning analytics and machine learning. Educational Technology & Society, 24(1), 236–249. https://www.jstor.org/stable/26977870
Mandinach, E. B., & Abrams, L. M. (2022). Data literacy and learning analytics. In C. Lang, A. F. Wise, A. Merceron, D. Gašević, & G. Siemens (Eds.), Handbook of learning analytics (2nd ed.) (pp. 196–204). https://doi.org/10.18608/hla22.019
Martins, R. M., Berge, E., Milrad, M., & Masiello, I. (2019). Visual learning analytics of multidimensional student behavior in self-regulated learning. Proceedings of the 14th European Conference on Technology Enhanced Learning (EC-TEL 2019), 16–19 September 2019, Delft, The Netherlands (pp. 737–741). Springer. https://doi.org/10.1007/978-3-030-29736-7_78
Martinez-Maldonado, R., Gašević, D., Echeverria, V., Fernandez Nieto, G., Swiecki, Z., & Buckingham Shum, S. (2021). What do you mean by collaboration analytics? A conceptual model. Journal of Learning Analytics, 8(1), 126–153. https://doi.org/10.18608/jla.2021.7227
Masiello, I., Mohseni, Z., Palma, F., Nordmark, S., Augustsson, H., & Rundquist, R. (2024). A current overview of the use of learning analytics dashboards. Education Sciences, 14(1), 82. https://doi.org/10.3390/educsci14010082
Matcha, W., Uzir, N. A., Gašević, D., & Pardo, A. (2020). A systematic review of empirical studies on learning analytics dashboards: A self-regulated learning perspective. IEEE Transactions on Learning Technologies, 13(2), 226–245. https://doi.org/10.1109/tlt.2019.2916802
McGrath, C., & Åkerfeldt, A. (2019). Educational technology (EdTech): Unbounded opportunities or just another brick in the wall? In A. Larsson & R. Teigland (Eds.), Digital transformation and public services: Societal impacts in Sweden and beyond (pp. 143–157). Routledge. https://www.taylorfrancis.com/chapters/oa-edit/10.4324/9780429319297-9/educational-technology-edtech-cormac-mcgrath-anna-%C3%A5kerfeldt
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G, & The PRISMA Group. (2010). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. International Journal of Surgery, 8(5), 336–341. https://doi.org/10.1016/j.ijsu.2010.02.007
Mohseni, Z., Martins, R. M., & Masiello, I. (2021). SAVis: A learning analytics dashboard with interactive visualization and machine learning. Proceedings of the Nordic Learning Analytics (Summer) Institute 2021, 23 August 2021, Stockholm, Sweden. CEUR Workshop Proceedings. https://ceur-ws.org/Vol-2985/paper2.pdf
Mohseni, Z., Martins, R. M., & Masiello, I. (2022). SBGTool v2.0: An empirical study on a similarity-based grouping tool for students’ learning outcomes. Data, 7(7), 98. https://doi.org/10.3390/data7070098
Molenaar, I., & Knoop-van Campen, C. A. N. (2019). How teachers make dashboard information actionable. IEEE Transactions on Learning Technologies, 12(3), 347–355. https://doi.org/10.1109/tlt.2018.2851585
Molenaar, I., Horvers, A., Dijkstra, R., & Baker, R. S. (2020). Personalized visualizations to promote young learners’ SRL: The learning path app. Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK ’20), 23–27 March 2020, Frankfurt, Germany (pp. 330–339). ACM Press. https://doi.org/10.1145/3375462.3375465
Norman, D. A. (2002). The design of everyday things. Basic Books.
Panadero, E., & Järvelä, S. (2015). Socially shared regulation of learning: A review. European Psychologist, 20(3), 190–203. https://doi.org/10.1027/1016-9040/a000226
Paulsen, L., & Lindsay, E. (2024). Learning analytics dashboards are increasingly becoming about learning and not just analytics: A systematic review. Education and Information Technologies. https://doi.org/10.1007/s10639-023-12401-4
Reich, J. (2022). Learning analytics and learning at scale. In C. Lang, A. F. Wise, A. Merceron, D. Gašević, & G. Siemens (Eds.), Handbook of learning analytics (2nd ed.) (pp. 188–195). Society for Learning Analytics Research. https://doi.org/10.18608/hla22.018
Rodríguez-Triana, M. J., Prieto, L. P., Dimitriadis, Y., de Jong, T., & Gillet, D. (2021). ADA for IBL: Lessons learned in aligning learning design and analytics for inquiry-based learning orchestration. Journal of Learning Analytics, 8(2), 22–50. https://doi.org/10.18608/jla.2021.7357
Rosmansyah, Y., Kartikasari, N., & Wuryandari, A. I. (2017). A learning analytics tool for monitoring and improving students’ learning process. Proceedings of the 6th International Conference on Electrical Engineering and Informatics (ICEEI), 25–27 November 2017, Langkawi, Malaysia. IEEE. https://doi.org/10.1109/ICEEI.2017.8312462
Sahin, M., & Ifenthaler, D. (2021). Visualizations and dashboards for learning analytics: A systematic literature review. In M. Sahin & D. Ifenthaler (Eds.), Visualizations and dashboards for learning analytics (pp. 3–22). Springer, Cham. https://doi.org/10.1007/978-3-030-81222-5_1
Sancenon, V., Wijaya, K., Wen, X. Y. S., Utama, D. A., Ashworth, M., Ng, K. H., Cheong, A., & Neo, Z. (2022). A new web-based personalized learning system improves students’ learning outcomes. International Journal of Virtual and Personal Learning Environments, 12(1). https://doi.org/10.4018/ijvple.295306
SBU. (2016). Evaluation and synthesis of studies using qualitative methods of analysis. Swedish Agency for Health Technology Assessment and Assessment of Social Services (SBU). https://www.sbu.se/globalassets/ebm/metodbok/sbuhandbook_qualitativemethodsofanalysis.pdf
Scardamalia, M. (2004). CSILE/Knowledge forum. In A. Kovalchick & K. Dawson (Eds.), Education and technology: An encyclopedia (pp. 183–192). ABC-CLIO. https://ikit.org/fulltext/CSILE_KF.pdf
Schildkamp, K., Karbautzki, L., & Vanhoof, J. (2014). Exploring data use practices around Europe: Identifying enablers and barriers. Studies in Educational Evaluation, 42, 15–24. https://doi.org/10.1016/j.stueduc.2013.10.007
Schwendimann, B. A., Rodríguez-Triana, M. J., Vozniuk, A., Prieto, L. P., Boroujeni, M. S., Holzer, A., Gillet, D., & Dillenbourg, P. (2017). Perceiving learning at a glance: A systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies, 10(1), 30–41. https://doi.org/10.1109/tlt.2016.2599522
Sedrakyan, G., Malmberg, J., Verbert, K., Järvelä, S., & Kirschner, P. A. (2020). Linking learning behavior analytics and learning science concepts: Designing a learning analytics dashboard for feedback to support learning regulation. Computers in Human Behavior, 107, 105512. https://doi.org/10.1016/j.chb.2018.05.004
Siemens, G., & Baker, R. S. J. d. (2012). Learning analytics and educational data mining: Toward communication and collaboration. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK ʼ12), 29 April–2 May 2012, Vancouver, BC, Canada (pp. 252–254). ACM Press. https://doi.org/10.1145/2330601.2330661
Sperling, K., Stenliden, L., Nissen, J., & Heintz, F. (2024). Behind the scenes of co-designing AI and LA in K–12 education. Postdigital Science and Education, 6(1), 321–341. https://doi.org/10.1007/s42438-023-00417-5
Tan, J. P.-L., Koh, E., Jonathan, C. R., & Yang, S. (2017). Learner dashboards a double-edged sword? Students’ sense-making of a collaborative critical reading and learning analytics environment for fostering 21st century literacies. Journal of Learning Analytics, 4(1), 117–140. https://doi.org/10.18608/jla.2017.41.7
Tlili, A., Hattab, S., Essalmi, F., Chen, N.-S., Huang, R., Kinshuk, Chang, M., & Burgos, D. (2021). A smart collaborative educational game with learning analytics to support English vocabulary teaching. International Journal of Interactive Multimedia and Artificial Intelligence, 6(6), 215–224. https://doi.org/10.9781/ijimai.2021.03.002
Tomkins, S., Ramesh, A., & Getoor, L. (2016). Predicting post-test performance from online student behaviour: A high school MOOC case study. In T. Barnes et al. (Eds.), Proceedings of the 9th International Conference on Educational Data Mining (EDM2016), 29 June–2 July 2016, Raleigh, NC, USA (pp. 239–246). International Educational Data Mining Society. https://www.educationaldatamining.org/EDM2016/proceedings/paper_123.pdf
Valle, N., Antonenko, P., Dawson, K., & Huggins‐Manley, A. C. (2021). Staying on target: A systematic literature review on learner‐facing learning analytics dashboards. British Journal of Educational Technology, 52(4), 1724–1748. https://doi.org/10.1111/bjet.13089
Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist, 57(10), 1500–1509. https://doi.org/10.1177/0002764213479363
Viberg, O., & Grönlund, Å. (2021). Desperately seeking the impact of learning analytics in education at scale: Marrying data analysis with teaching and learning. In J. Liebowitz (Ed.), Online learning analytics. Auerbach Publications. https://doi.org/10.1201/9781003194620
Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in Human Behavior, 89, 98–110. https://doi.org/10.1016/j.chb.2018.07.027
Vieira, C., Parsons, P., & Byrd, V. (2018). Visual learning analytics of educational data: A systematic literature review and research agenda. Computers & Education, 122, 119–135. https://doi.org/10.1016/j.compedu.2018.03.018
Vrugte, J. T., & Eshuis, E. H. (2022). Visualization and reflection to scaffold time-management in a computer-based learning environment. Proceedings of the 16th International Conference of the Learning Sciences (ISLS ’22), 6–10 June 2022, Hiroshima, Japan (pp. 1389–1392). International Society of the Learning Sciences.
Vuorikari, R., Kluzer, S., & Punie, Y. (2022). DigComp 2.2, The digital competence framework for citizens: With new examples of knowledge, skills and attitudes. Publications Office of the European Union. https://doi.org/10.2760/115376
Wang, P., Chen, G., Tong, Y., & Yang, C. (2022). Post-flipped classrooms: Designing a video-based visualization learning approach for supporting emergency remote teaching. Proceedings of the 15th International Conference on Computer-Supported Collaborative Learning (CSCL 2022), 6–10 June 2022, Hiroshima, Japan (pp. 282–289). International Society of the Learning Sciences.
Watts, T. W., Duncan, G. J., Siegler, R. S., & Davis-Kean, P. E. (2014). What’s past is prologue: Relations between early mathematics knowledge and high school achievement. Educational Researcher, 43(7), 352–360. https://doi.org/10.3102/0013189x14553660
Wilson, M., & Lehrer, R. (2021). Improving learning: Using a learning progression to coordinate instruction and assessment. Frontiers in Education, 6, 654212. https://doi.org/10.3389/feduc.2021.654212
Yang, Y., & Song, Y. (2022). Understanding primary students’ self-regulated vocabulary learning behaviours on a mobile app via learning analytics and their associated outcomes: A case study. Journal of Computers in Education, 10(3), 469–498. https://doi.org/10.1007/s40692-022-00251-x
Yeung, K. L., Carpenter, S. K., & Corral, D. (2021). A comprehensive review of educational technology on objective learning outcomes in academic contexts. Educational Psychology Review, 33(4), 1583–1630. https://doi.org/10.1007/s10648-020-09592-4
Zamecnik, A., Kovanović, V., Grossmann, G., Joksimović, S., Jolliffe, G., Gibson, D., & Pardo, A. (2022). Team interactions with learning analytics dashboards. Computers & Education, 185, 104514. https://doi.org/10.1016/j.compedu.2022.104514
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
