A Systematic Review of Learning Analytics
Incorporated Instructional Interventions on Learning Management Systems
DOI:
https://doi.org/10.18608/jla.2023.8093Keywords:
learning management system, learning analytics interventions, instructional interventions, systematic review, research paperAbstract
The learning management system (LMS) is widely used in educational settings to support teaching and learning practices. The usage log data, generated by both learners and instructors, enables the development and implementation of learning analytics (LA) interventions aimed at facilitating teaching and learning activities. To examine the current status of the development and empirical impacts of learning analytics–incorporated interventions within LMSs on improving teaching and learning practices, we conducted a systematic review that examined 27 articles published from 2012 through 2023. The outcomes of this review provided valuable insights into the design and development of learning analytics–incorporated interventions implemented on LMSs and empirical evidence of the impacts of these interventions, along with implications to inform future design and applications.
References
Abazi-Bexheti, L., Kadriu, A., Apostolova-Trpkovska, M., Jajaga, E., & Abazi-Alili, H. (2018). LMS solution: Evidence of Google Classroom usage in higher education. Business Systems Research Journal, 9(1), 31–43. https://doi.org/10.2478/bsrj-2018-0003
Abdul Rahman, M. F., Umar, I. N., & Abdullah, Z. (2017). The effects of students’ characteristics on engagement in a learning management system environment. Forum Komunikasi, 12(2), 21–36. https://forumkomunikasi.uitm.edu.my/images/FK2017Vol12No2/FK2017-chapt-2.pdf
Afify, M. K. (2019). The influence of group size in the asynchronous online discussions on the development of critical thinking skills, and on improving students’ performance in online discussion forums. International Journal of Emerging Technologies in Learning, 14(5), 132–152. https://doi.org/10.3991/ijet.v14i05.9351
Afzaal, M., Nouri, J., Zia, A., Papapetrou, P., Fors, U., Wu, Y., Li, X., & Weegar, R. (2021). Explainable AI for data-driven feedback and intelligent action recommendations to support students self-regulation. Frontiers in Artificial Intelligence, 4, 723447. https://doi.org/10.3389/frai.2021.723447
Alhazmi, A., Imtiaz, A., Alhammadi, F., & Kaed, E. (2021). Success and failure aspects of LMS in e-learning systems. International Journal of Interactive Mobile Technologies, 15(11), 133–147. https://doi.org/10.3991/ijim.v15i11.20805
Aljohani, N. R., Daud, A., Abbasi, R. A., Alowibdi, J. S., Basheri, M., & Aslam, M. A. (2019). An integrated framework for course adapted student learning analytics dashboard. Computers in Human Behavior, 92, 679–690. https://doi.org/10.1016/j.chb.2018.03.035
Alotaibi, K., Tohmaz, R., & Jabak, O. (2017). The relationship between self-regulated learning and academic achievement for a sample of community college students at King Saud University. Education Journal, 6(1), 28–37. http://dx.doi.org/10.2139/ssrn.3444622
Al-Shaikhli, D., Jin, L., Porter, A., & Tarczynski, A. (2022). Visualizing weekly learning outcomes (VWLO) and the intention to continue using a learning management system (CIU): The role of cognitive absorption and perceived learning self-regulation. Education and Information Technologies, 27(3), 2909–2937. https://doi.org/10.1007/s10639-021-10703-z
Altinpulluk, H., & Kesim, M. (2021). A systematic review of the tendencies in the use of learning management systems. Turkish Online Journal of Distance Education, 22(3), 40–54. http://dx.doi.org/10.17718/tojde.961812
Ahn, J., Campos, F., Hays, M., & Digiacomo, D. (2019). Designing in context: Reaching beyond usability in learning analytics dashboard design. Journal of Learning Analytics, 6(2), 70–85. https://doi.org/10.18608/jla.2019.62.5
Araka, E., Maina, E., Gitonga, R., & Oboko, R. (2020). Research trends in measurement and intervention tools for self-regulated learning for e-learning environments: Systematic review (2008–2018). Research and Practice in Technology Enhanced Learning, 15, 6. http://dx.doi.org/10.1186/s41039-020-00129-5
Arnold, K. E., & Pistilli, M. D. (2012). Course Signals at Purdue: Using learning analytics to increase student success. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK ʼ12), 29 April–2 May 2012, Vancouver, BC, Canada (pp. 267–270). ACM Press. https://doi.org/10.1145/2330601.2330666
Asha, S., & Chellappan, C. (2011). Voice activated e-learning system for the visually impaired. International Journal of Computer Applications, 14(7), 42–51. http://dx.doi.org/10.5120/1892-2514
Atif, A., Richards, D., Liu, D., & Bilgin, A. A. (2020). Perceived benefits and barriers of a prototype early alert system to detect engagement and support ‘at-risk’ students: The teacher perspective. Computers & Education, 156, 103954. https://doi.org/10.1016/j.compedu.2020.103954
Balkaya, S., & Akkucuk, U. (2021). Adoption and use of learning management systems in education: The role of playfulness and self-management. Sustainability, 13(3), 1127. https://doi.org/10.3390/su13031127
Bodily, R., & Verbert, K. (2017). Review of research on student-facing learning analytics dashboards and educational recommender systems. IEEE Transactions on Learning Technologies, 10(4), 405–418. https://doi.org/10.1109/TLT.2017.2740172
Chanifah, S., Andreswari, R., & Fauzi, R. (2021). Analysis of student learning pattern in learning management system (LMS) using heuristic mining: A process mining approach. Proceedings of the 3rd International Conference on Electronics Representation and Algorithm (ICERA), 29 July 2021, Virtual (pp. 121–125). IEEE. https://doi.org/10.1109/ICERA53111.2021.9538654
Chen, C.-C., Huang, C., Gribbins, M., & Swan, K. (2018). Gamify online courses with tools built into your learning management system (LMS) to enhance self-determined and active learning. Online Learning, 22(3), 41–54. https://doi.org/10.24059/olj.v22i3.1466
Chung, J., Mundy, M. E., & McKenzie, S. (2022). A self-managed online mindfulness program in a university-wide learning management system orientation site: A real-world ecological validation study. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.869765
Costa, L. A., Pereira Sanches, L. M., Rocha Amorim, R. J., Nascimento Salvador, L. D., & Santos Souza, M. V. D. (2020). Monitoring academic performance based on learning analytics and ontology: A systematic review. Informatics in Education, 19(3), 361–397. https://doi.org/10.15388/infedu.2020.17
Creswell, J. W., & Creswell J. D. (2017). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Sage Publications.
Daimari, D., Narzary, M., Mazumdar, N., & Nag, A. (2021). A machine learning based book availability prediction model for library management system. Library Philosophy and Practice, 4982. https://digitalcommons.unl.edu/libphilprac/4982
Davies, R., Nyland, R., Bodily, R., Chapman, J., Jones, B., & Young, J. (2017). Designing technology-enabled instruction to utilize learning analytics. TechTrends, 61(2), 155–161. https://doi.org./10.1007/s11528-016-0131-7
de Lima, D. P. R., Gerosa, M. A., Conte, T. U., & de M. Netto, J. F. (2019). What to expect, and how to improve online discussion forums: The instructors’ perspective. Journal of Internet Services and Applications, 10, 22. https://doi.org/10.1186/s13174-019-0120-0
Ferdiánová, V. (2017). GeoGebra materials for LMS Moodle focused Monge on projection. Electronic Journal of e-Learning, 15(3), 259–268. https://academic-publishing.org/index.php/ejel/article/view/1837
Fung, C. Y., Abdullah, M. N. L. Y., & Hashim, S. (2019). Improving self-regulated learning through personalized weekly e-learning journals: A time series quasi-experimental study. e-Journal of Business Education and Scholarship of Teaching, 13(1), 30–45. https://eric.ed.gov/?id=EJ1239150
García-Martín, J., & García-Sánchez, J.-N. (2018). The instructional effectiveness of two virtual approaches: Processes and product. Revista de Psicodidáctica (English ed.), 23(2), 117–127. https://doi.org/10.1016/j.psicoe.2018.02.003
García-Murillo, G., Novoa-Hernández, P., & Rodríguez, R. S. (2020). Technological satisfaction about Moodle in higher education: A meta-analysis. IEEE Revista Iberoamericana de Tecnologias Del Aprendizaje, 15(4), 281–290. https://doi.org/10.1109/RITA.2020.3033201
Gašević, D., Jovanovic, J., Pardo, A., & Dawson, S. (2017). Detecting learning strategies with analytics: Links with self-reported measures and academic performance. Journal of Learning Analytics, 4(2), 113–128. https://doi.org/10.18608/jla.2017.42.10
Goomas, D., & Czupryn, K. (2021). Using a learning management system common template in teaching adult basic education: Opportunities and challenges. Community College Journal of Research and Practice, 45(3), 227–230. https://doi.org/10.1080/10668926.2019.1669229
Gutman, M. (2017). Facilitating pre-service teachers to develop regulation of cognition with learning management system. Educational Media International, 54(3), 199–214. http://dx.doi.org/10.1080/09523987.2017.1385164
Hariri, M. M. (2014). Students’ perceptions of the utilization of learning management system (LMS) features: A case study of a geology course at KFUPM, Saudi Arabia. International Journal of Technology Diffusion, 5(4), 50–62. https://doi.org/10.4018/ijtd.2014100104
Ismail, S. N., Hamid, S., Ahmad, M., Alaboudi, A., & Jhanjhi, N. (2021). Exploring students engagement towards the learning management system (LMS) using learning analytics. Computer Systems Science and Engineering, 37(1), 73–87. https://doi.org/10.32604/csse.2021.015261
Jayashanka, R., Hettiarachchi, E., & Hewagamage, K. P. (2022). Technology enhanced learning analytics dashboard in higher education. The Electronic Journal of e-Learning, 20(2), 151–170. https://doi.org/10.34190/ejel.20.2.2189
Jivet, I., Scheffel, M., Specht, M., & Drachsler, H. (2018). License to evaluate: Preparing learning analytics dashboards for educational practice. Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK ’18), 5–9 March 2018, Sydney, NSW, Australia (pp. 31–40). ACM Press. https://doi.org/10.1145/3170358.3170421
Karaoglan Yilmaz, F. G. (2022a). The effect of learning analytics assisted recommendations and guidance feedback on students’ metacognitive awareness and academic achievements. Journal of Computing in Higher Education, 34(2), 396–415. https://doi.org/10.1007/s12528-021-09304-z
Karaoglan Yilmaz, F. G. (2022b). Utilizing learning analytics to support students’ academic self-efficacy and problem-solving skills. The Asia-Pacific Education Researcher, 31(2), 175–191. https://doi.org/10.1007/s40299-020-00548-4
Karaoglan Yilmaz, F. G., & Yilmaz, R. (2022). Learning analytics intervention improves students’ engagement in online learning. Technology, Knowledge and Learning, 27(2), 449–460. https://doi.org/10.1007/s10758-021-09547-w
Khiat, H., & Vogel, S. (2022). A self-regulated learning management system: Enhancing performance, motivation and reflection in learning. Journal of University Teaching & Learning Practice, 19(2), 43–59. https://doi.org/10.53761/1.19.2.4
Klein, C., Lester, J., Nguyen, T., Justen, A., Rangwala, H., & Johri, A. (2019). Student sensemaking of learning analytics dashboard interventions in higher education. Journal of Educational Technology Systems, 48(1), 130–154. https://doi.org/10.1177/0047239519859854
Knobbout, J., & Van Der Stappen, E. (2020). Where is the learning in learning analytics? A systematic literature review on the operationalization of learning-related constructs in the evaluation of learning analytics interventions. IEEE Transactions on Learning Technologies, 13(3), 631–645. https://doi.org/10.1109/tlt.2020.2999970
Laeeq, K., & Memon, Z. A. (2021). Scavenge: An intelligent multi-agent based voice-enabled virtual assistant for LMS. Interactive Learning Environments, 29(6), 954–972. https://doi.org/10.1080/10494820.2019.1614634
Lallé, S., Conati, C., Azevedo, R., Mudrick, N., & Taub, M. (2017). On the influence on learning of student compliance with prompts fostering self-regulated learning. In X. Hu, T. Barnes, A. Hershkovitz, & L. Paquette (Eds.), Proceedings of the 10th International Conference on Educational Data Mining, 25–28 June 2017, Wuhan, China (pp. 120–127). International Educational Data Mining Society. https://educationaldatamining.org/EDM2017/proc_files/proceedings.pdf
Lang, C., Siemens, G., Wise, A., & Gašević, D. (Eds.). (2017). The handbook of learning analytics. SOLAR. https://doi.org/10.18608/hla17
Larrabee Sønderlund, A., Hughes, E., & Smith, J. (2019). The efficacy of learning analytics interventions in higher education: A systematic review. British Journal of Educational Technology, 50(5), 2594–2618. https://doi.org/10.1111/bjet.12720
Lima, D. A., & Isotani, S. (2021). Constraints, effectiveness and solutions in using Google Classroom as a learning management system during Covid-19 pandemic: A systematic literature review. Proceedings of the 16th Latin American Conference on Learning Technologies (LACLO 2021), 19–21 October 2021, Arequipa, Peru (pp. 38–43). IEEE. https://doi.org/10.1109/LACLO54177.2021.00012
Liu, D. Y.-T., Froissard, J.-C., Richards, D., & Atif, A. (2015). An enhanced learning analytics plugin for Moodle: Student engagement and personalized intervention. In T. Reiners, B. R. von Konsky, D. Gibson, V. Chang, L. Irving, & K. Clarke (Eds.), Globally Connected, Digitally Enabled: Proceedings of the Australasian Society for Computers in Learning and Tertiary Education Conference (ASCILITE2015), 29 November–2 December 2015, Perth, Australia (pp. FP:168–FP:177). Curtin University.
Liu, M., Han, S., Shao, P., Cai, Y., & Pan, Z. (2021). The current landscape of research and practice on visualizations and dashboards for learning analytics. In M. Sahin & D. Ifenthaler (Eds.), Visualizations and Dashboards for Learning Analytics (pp. 23–46). Springer Cham. https://doi.org/10.1007/978-3-030-81222-5_2
Lonn, S., Aguilar, S. J., & Teasley, S. D. (2015). Investigating student motivation in the context of a learning analytics intervention during a summer bridge program. Computers in Human Behavior, 47, 90–97. https://doi.org/10.1016/j.chb.2014.07.013
Lu, O. H. T., Huang, J. C. H., Huang, A. Y. Q., & Yang, S. J. H. (2017). Applying learning analytics for improving students engagement and learning outcomes in an MOOCs enabled collaborative programming course. Interactive Learning Environments, 25(2), 220–234. https://doi.org/10.1080/10494820.2016.1278391
Mangaroska, K., & Giannakos, M. (2019). Learning analytics for learning design: A systematic literature review of analytics-driven design to enhance learning. IEEE Transactions on Learning Technologies, 12(4), 516–534. https://doi.org/10.1109/TLT.2018.2868673
Miah, S. J., Miah, M., & Shen, J. (2020). Editorial note: Learning management systems and big data technologies for higher education. Education and Information Technologies, 25(2), 725–730. https://doi.org/10.1007/s10639-020-10129-z
Miles, M. B., Huberman, A. M., & Saldaña, J. (2013). Qualitative data analysis: A methods sourcebook (3rd ed.). Sage Publications.
Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Journal of Clinical Epidemiology, 62(10), 1006–1012. https://doi.org/10.1016/j.jclinepi.2009.06.005
Muljana, P. S., Dabas, C. S., & Luo, T. (2023). Examining the relationships among self-regulated learning, homework timeliness, and course achievement: A context of female students learning quantitative topics. Journal of Research on Technology in Education, 55(2), 143–162. https://doi.org/10.1080/15391523.2021.1936703
Mwalumbwe, I., & Mtebe, J. S. (2017). Using learning analytics to predict students’ performance in Moodle learning management system: A case of Mbeya University of Science and Technology. The Electronic Journal of Information Systems in Developing Countries, 79(1), 1–13. https://doi.org/10.1002/j.1681-4835.2017.tb00577.x
Na, K. S., & Tasir, Z. (2017). A systematic review of learning analytics intervention contributing to student success in online learning. Proceedings of the 2017 International Conference on Learning and Teaching in Computing and Engineering (LaTICE 2017), 20–23 April 2017, Hong Kong, (pp. 62–68). IEEE. https://doi.org/10.1109/LaTiCE.2017.18
Odhiambo, B. A., Okeyo, G., & Cheruiyot, W. (2017). Framework for improving usability of learning management systems by integrating pedagogical agent. International Journal of Computer Applications, 166(8), 7–16. https://doi.org/10.5120/ijca2017914087
Ozdemir, D., Opseth, H. M., & Taylor, H. (2020). Leveraging learning analytics for student reflection and course evaluation. Journal of Applied Research in Higher Education, 12(1), 27–37. https://doi.org/10.1108/JARHE-11-2018-0253
Pan, Z., Li, C., & Liu, M. (2020). Learning analytics dashboard for problem-based learning. Proceedings of the 7th ACM Conference on Learning @ Scale (L@S 2020), 12–14 August 2020, Virtual Event USA (pp. 393–396). ACM Press. https://doi.org/10.1145/3386527.3406751
Poon, L. K. M., Kong, S.-C., Wong, M. Y. W., & Yau, T. S. H. (2017). Mining sequential patterns of students’ access on learning management system. In Y. Tan, H. Takagi, & Y. Shui (Eds.), Data Mining and Big Data: Second International Conference (DMBD 2017), 27 July–1 August 2017, Fukuoka, Japan (pp. 191–198). Springer Cham. https://doi.org/10.1007/978-3-319-61845-6_20
Pozdniakov, S., Martinez-Maldonado, R., Tsai, Y.-S., Cukurova, M., Bartindale, T., Chen, P., Marshall, H., Richardson, D., & Gašević, D. (2022). The question-driven dashboard: How can we design analytics interfaces aligned to teachers’ inquiry? Proceedings of the 12th International Conference on Learning Analytics and Knowledge (LAK ’22), 21–25 March 2022, Online (pp. 175–185). ACM Press. https://doi.org/10.1145/3506860.3506885
Prahani, B. K., Alfin, J., Fuad, A. Z., Saphira, H. V., Hariyono, E., & Suprapto, N. (2022). Learning management system (LMS) research during 1991–2021: How technology affects education. International Journal of Emerging Technologies in Learning, 17(17), 28–49. https://doi.org/10.3991/ijet.v17i17.30763
Saadati, Z., Zeki, C. P., & Vatankhah Barenji, R. (2023). On the development of blockchain-based learning management system as a metacognitive tool to support self-regulation learning in online higher education. Interactive Learning Environments, 31(5), 3148–3171. https://doi.org/10.1080/10494820.2021.1920429
Safsouf, Y., Mansouri, K., & Poirier, F. (2021). TaBAT: Design and experimentation of a learning analysis dashboard for teachers and learners. Journal of Information Technology Education: Research, 20, 331–350. https://doi.org/10.28945/4820
Şahin, M., & Yurdugül, H. (2019). An intervention engine design and development based on learning analytics: The intelligent intervention system (In2S). Smart Learning Environments, 6(1), 18. https://doi.org/10.1186/s40561-019-0100-7
Santoso, L. W. (2020). Predicting student performance in higher education using multi-regression models. Telkomnika Telecommunication, Computing, Electronics and Control, 18(3), 1354–1360. http://doi.org/10.12928/telkomnika.v18i3.14802
Saroha, K., & Mehta, P. (2016). Analysis and evaluation of learning management system using data mining techniques. Proceedings of the 2016 International Conference on Recent Trends in Information Technology (ICRTIT), 8–9 April 2016, Chennai, India (pp. 1–5). IEEE. https://doi.org/10.1109/ICRTIT.2016.7569542
Schwendimann, B. A., Rodríguez-Triana, M. J., Vozniuk, A., Prieto, L. P., Boroujeni, M. S., Holzer, A., Gillet, D., & Dillenbourg, P. (2017). Perceiving learning at a glance: A systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies, 10(1), 30–41. https://doi.org/10.1109/TLT.2016.2599522
Sedrakyan, G., Malmberg, J., Verbert, K., Järvelä, S., & Kirschner, P. A. (2020). Linking learning behaviour analytics and learning science concepts: Designing a learning analytics dashboard for feedback to support learning regulation. Computers in Human Behaviour, 107, 105512. https://doi.org/10.1016/j.chb.2018.05.004
Setiadi, P. M., Alia, D., Sumardi, S., Respati, R., & Nur, L. (2021). Synchronous or asynchronous? Various online learning platforms studied in Indonesia 2015–2020. Journal of Physics: Conference Series, 1987, 012016. https://doi.org/10.1088/1742-6596/1987/1/012016
Shine, B., & Heath, S. E. (2020). Techniques for fostering self-regulated learning via learning management systems in on-campus and online courses. Journal of Teaching and Learning with Technology, 9(1). https://doi.org/10.14434/jotlt.v9i1.29014
Su, C.-Y., Li, Y.-H., & Chen, C.-H. (2021). Understanding the behavioural patterns of university teachers toward using a learning management system. International Journal of Emerging Technologies in Learning, 16(14), 129–145. https://doi.org/10.3991/ijet.v16i14.22685
Tabuenca, B., Kalz, M., Drachsler, H., & Specht, M. (2015). Time will tell: The role of mobile learning analytics in self-regulated learning. Computers & Education, 89, 53–74. https://doi.org/10.1016/j.compedu.2015.08.004
Tan, T. K., & Samavedham, L. (2022). The learning process matter: A sequence analysis perspective of examining procrastination using learning management system. Computers and Education Open, 3, 100112. https://doi.org/10.1016/j.caeo.2022.100112
Tempelaar, D., Rienties, B., & Nguyen, Q. (2021). The contribution of dispositional learning analytics to precision education. Educational Technology & Society, 24(1), 109–122. https://www.jstor.org/stable/26977861
Verbert, K., Ochoa, X., De Croon, R., Dourado, R. A., & De Laet, T. (2020). Learning analytics dashboards: The past, the present and the future. Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK ’20), 23–27 March 2020, Frankfurt, Germany (pp. 35–40). ACM Press. https://doi.org/10.1145/3375462.3375504
Wan, H., Zhong, Z., Tang, L., & Gao, X. (2023). Pedagogical interventions in SPOCs: Learning behaviour dashboards and knowledge tracing support exercise recommendation. IEEE Transactions on Learning Technologies, 16(3), 431–442. https://doi.org/10.1109/TLT.2023.3242712
Willans, F., Fonolahi, A., Buadromo, R., Bryce, T., Prasad, R., & Kumari, S. (2019). Fostering and evaluating learner engagement with academic literacy support: Making the most of Moodle. Journal of University Teaching & Learning Practice, 16(4), 5. https://doi.org/10.53761/1.16.4.5
Wintoro, P. B., & Pratama, M. (2022). Course clustering in Moodle based learning management system using unsupervised learning. AIP Conference Proceedings, 2563(1), 030006. https://doi.org/10.1063/5.0103298
Wong, G. K. W., Li, Y. K., & Lai, X. (2021). Visualizing the learning patterns of topic-based social interaction in online discussion forums: An exploratory study. Educational Technology Research and Development, 69(5), 2813–2843. https://doi.org/10.1007/s11423-021-10040-5
Xin, N. S., Shibghatullah, A. S., Subaramaniam, K. A., & Wahab, M. H. A. (2021). A systematic review for online learning management system. Journal of Physics: Conference Series, 1874, 012030. https://doi.org/10.1088/1742-6596/1874/1/012030
Zhang, F., & She, M. (2021). Design of English reading and learning management system in college education based on artificial intelligence [Unpublished article].
Zheng, L., Niu, J., & Zhong, L. (2022). Effects of a learning analytics‐based real‐time feedback approach on knowledge elaboration, knowledge convergence, interactive relationships and group performance in CSCL. British Journal of Educational Technology, 53(1), 130–149. https://doi.org/10.1111/bjet.13156
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST