How Can Predictive Learning Analytics and Motivational Interventions Increase Student Retention and Enhance Administrative Support in Distance Education?
DOI:
https://doi.org/10.18608/jla.2020.72.4Keywords:
predictive learning analytics, motivational interventions, student support, distance learning, randomized control trialAbstract
Despite the potential of Predictive Learning Analytics (PLAs) to identify students at risk of failing their studies, research demonstrating effective application of PLAs to higher education is relatively limited. The aims of this study are 1) to identify whether and how PLAs can inform the design of motivational interventions and 2) to capture the impact of those interventions on student retention at the Open University UK. A predictive model — the Student Probabilities Model (SPM) — was used to predict the likelihood of a student remaining in a course at the next milestone and eventually completing it. Undergraduate students (N=630) with a low probability of completing their studies were randomly allocated into the control (n=312) and intervention groups (n=318), and contacted by the university Student Support Teams (SSTs) using a set of motivational interventions such as text, phone, and email. The results of the randomized control trial showed statistically significant better student retention outcomes for the intervention group, with the proposed intervention deemed effective in facilitating course completion. The intervention also improved the administration of student support at scale and low cost.
References
Arnold, K., & Pistilli, M. (2012). Course Signals at Purdue: Using learning analytics to increase student success. In S. Buckingham Shum, D. Gašević, & R. Ferguson (Eds.), Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK ʼ12), 29 April–2 May 2012, Vancouver, BC, Canada (pp. 267–270). New York: ACM.
Beard, J., Stevens, C., & Welch, A. (2017). Undergraduate MILLS framework 2017/18. Student Support Centre for Excellence, Open University, UK.
Calvert, C. (2014). Developing a model and applications for probabilities of student success: A case study of predictive analytics. Open Learning: The Journal of Open, Distance and e-Learning, 29(2), 160–173. https://dx.doi.org/10.1080/02680513.2014.931805
Castleman, B. L., & Page, L. C. (2015). Summer nudging: Can personalized text messages and peer mentor outreach increase college going among low-income high school graduates? Journal of Economic Behavior and Organization, 115, 144–160. https://dx.doi.org/10.1016/j.jebo.2014.12.008
Conijn, R., Snijders, C., Kleingeld, A., & Matzat, U. (2017). Predicting student performance from LMS data: A comparison of 17 blended courses using Moodle LMS. IEEE Transactions on Learning Technologies, 10(1), 17–29. https://dx.doi.org/10.1109/tlt.2016.2616312
Dawson, S., Jovanovic, J., Gašević, D., & Pardo, A. (2017). From prediction to impact: Evaluation of a learning analytics retention program. Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 474–478). New York: ACM.
Dolata, U. (2013). The transformative capacity of new technologies: A theory of sociotechnical change. London: Routledge.
Ferguson, R., Brasher, A., Cooper, A., Hillaire, G., Mittelmeier, J., Rienties, B., ... Vuorikari, R. (2016). A European framework for action on learning analytics. In R. Vuorikari & J. Castaño-Muñoz (Eds.), Research evidence of the use of learning analytics: Implications for education policy (pp. 1–152). Luxembourg: Publications Office of the European Union.
Ferguson, R., & Clow, D. (2017). Where is the evidence? Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 56–65). New York: ACM.
Fozdar, B., & Kumar, L. (2007). Mobile learning and student retention. International Review of Research in Open and Distance Learning, 8(2), 1–18. https://dx.doi.org/10.4103/0973-029X.102522
Harley, J. M., Carter, C. K., Papaionnou, N., Bouchet, F., Landis, R. S., Azevedo, R., & Karabachian, L. (2016). Examining the predictive relationship between personality and emotion traits and students’ agent-directed emotions: Towards emotionally-adaptive agent-based learning environments. User Modeling and User-Adapted Interaction, 26(2–3), 177–219. https://dx.doi.org/10.1007/s11257-016-9169-7
Herodotou, C., Rienties, B., Hlosta, M., Boroowa, A., Mangafa, C., & Zdrahal, Z. (2020). The scalable implementation of predictive learning analytics at a distance learning university: Insights from a longitudinal case study. The Internet and Higher Education, 45. https://dx.doi.org/10.1016/j.iheduc.2020.100725
Herodotou, C., Hlosta, M., Boroowa, A., Rienties, B., Zdrahal, Z., & Mangafa, C. (2019a). Empowering online teachers through predictive learning analytics. British Journal of Educational Technology, 50(6), 3064–3079. https://bera-journals.onlinelibrary.wiley.com/doi/abs/10.1111/bjet.12853
Herodotou, C., Rienties, B., Boroowa, A., Zdrahal, Z., & Hlosta, M. (2019b). A large-scale implementation of predictive learning analytics in higher education: The teachers’ role and perspective. Educational Technology Research and Development, 67(5), 1273–1306. https://dx.doi.org/10.1007/s11423-019-09685-0
Hlosta, M., Zdrahal, Z., & Zendulka, J. (2017). Ouroboros: Early identification of at-risk students without models based on legacy data. Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 6–15). New York: ACM.
Hung, I. C., Chao, K. J., Lee, L., & Chen, N. S. (2013). Designing a robot teaching assistant for enhancing and sustaining learning motivation. Interactive Learning Environments, 21(2), 156–171. https://dx.doi.org/10.1080/10494820.2012.705855
Inkelaar, T., & Simpson, O. (2015). Challenging the “distance education deficit” through “motivational emails.” Open Learning, 30(2), 152–163. https://dx.doi.org/10.1080/02680513.2015.1055718
Jayaprakash, S. M., Moody, E. W., Lauría, E. J. M., Regan, J. R., & Baron, J. D. (2014). Early alert of academically at-risk students: An open source analytics initiative. Journal of Learning Analytics, 1(1), 6–47. https://dx.doi.org/10.18608/jla.2014.11.3
Kim, C. (2008). Using email to enable e3 (effective, efficient, and engaging) learning. Distance Education, 29(2), 187–198. https://dx.doi.org/10.1080/01587910802154988
Kizilcec, R. F., Pérez-Sanagustín, M., & Maldonado, J. (2017). Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses. Computers & Education, 104, 18–33. https://dx.doi.org/10.1016/j.compedu.2016.10.001
Kovanović, V., Gašević, D., Dawson, S., Joksimović, S., Baker, R. S., & Hatala, M. (2015). Penetrating the black box of time-on-task estimation. Proceedings of the 5th International Conference on Learning Analytics and Knowledge (LAK ʼ15), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 184–193). New York: ACM.
Lukosius, V., Pennington, J. B., & Olorunniwo, F. O. (2013). How students’ perceptions of support systems affect their intentions to drop out or transfer out of college. Review of Higher Education & Self-Learning, 6(18), 209–221.
Milem, J. F., & Berger, J. B. (1997). A modified model of college student persistence: Exploring the relationship between Astin’s theory of involvement and Tinto’s theory of student departure. Journal of College Student Development, 38(4), 387–400.
Moore, M. G. (1997). Theory of transactional distance. In D. Keegan (Ed.), Theoretical principles of distance education (pp. 22–38). London: Routledge.
Naismith, L. (2007). Using text messaging to support administrative communication in higher education. Active Learning in Higher Education, 8(2), 155–171. https://dx.doi.org/10.1177/1469787407078000
Ryan, G. S., & Davies, F. (2016). Development and evaluation of an online, interactive information and advice tool for pre-registration nursing students. Nurse Education in Practice, 17, 214–222. https://dx.doi.org/10.1016/j.nepr.2016.01.002
Scheffel, M., Drachsler, H., de Kraker, J., Kreijns, K., Slootmaker, A., & Specht, M. (2017). Widget, widget on the wall, am I performing well at all? IEEE Transactions on Learning Technologies, 10(1), 42–52. https://dx.doi.org/10.1109/tlt.2016.2622268
Sclater, N. (2017). Learning analytics explained. New York: Routledge.
Sclater, N., Peasgood, A., & Mullan, J. (2016). Learning analytics in higher education: A review of UK and international practice. Retrieved from https://go.aws/328xLVv
Seidel, E., & Kutieleh, S. (2017). Using predictive analytics to target and improve first-year student attrition. Australian Journal of Education, 61(2), 200–218. https://dx.doi.org/10.1177/0004944117712310
Shea, P., & Bidjerano, T. (2014). Does online learning impede degree completion? A national study of community college students. Computers & Education, 75(Supplement C), 103–111. https://dx.doi.org/10.1016/j.compedu.2014.02.009
Simpson, O. (2013). Student retention in distance education: Are we failing our students? Open Learning, 28(2), 105–119. https://dx.doi.org/10.1080/02680513.2013.847363
Tempelaar, D., Rienties, B., Mittelmeier, J., & Nguyen, Q. (2018). Student profiling in a dispositional learning analytics application using formative assessment. Computers in Human Behavior, 78, 408–420. https://dx.doi.org/10.1016/j.chb.2017.08.010
van Leeuwen, A., Janssen, J., Erkens, G., & Brekelmans, M. (2015). Teacher regulation of cognitive activities during student collaboration: Effects of learning analytics. Computers & Education, 90, 80–94. https://dx.doi.org/10.1016/j.compedu.2015.09.006
van Ameijde, J., Weller, M., & Cross, S. (2018). Learning design for student retention. Journal of Perspectives in Applied Academic Practice, 6(2), 41–50. https://dx.doi.org/10.14297/jpaap.v6i2.318
Wentzel, K. R., & Wigfield, A. (2007). Motivational interventions that work: Themes and remaining issues, 1520, 261–271. https://dx.doi.org/10.1080/00461520701621103
West, D., Luzeckyj, A., Searle, B., Toohey, D., & Price, R. (2018). The use of learning analytics to support improvements in teaching practice. Innovative Research Universities. Melbourne, Australia.
Wilcock, C. (2011). The Open University in Wales: Proactive student support. In L. Thomas & C. Jamieson-Bal (Eds.), Engaging students to improve student retention and success in higher education in Wales. York, UK: Higher Education Academy.
Wise, A. F. (2014). Designing pedagogical interventions to support student use of learning analytics. Proceedings of the 4th International Conference on Learning Analytics and Knowledge (LAK ʼ14), 24–28 March 2014, Indianapolis, IN, USA (pp. 203–211). New York: ACM.
Worsley, M., & Blikstein, P. (2015). Leveraging multimodal learning analytics to differentiate student learning strategies. Proceedings of the 5th International Conference on Learning Analytics and Knowledge (LAK ʼ15), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 360–367). New York: ACM.
Wright, M. C., McKay, T., Hershock, C., Miller, K., & Tritz, J. (2014). Better than expected: Using learning analytics to promote student success in gateway science. Change: The Magazine of Higher Learning, 46(1), 28–34. https://dx.doi.org/10.1080/00091383.2014.867209
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2020 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
