Predicting University Students’ Exam Performance Using a Model-Based Adaptive Fact-Learning System
DOI:
https://doi.org/10.18608/jla.2021.6590Keywords:
adaptive learning, technology-enhanced learning, cognitive tutor, cognitive model, rate of forgetting, academic achievement, research paperAbstract
Modern educational technology has the potential to support students to use their study time more effectively. Learning analytics can indicate relevant individual differences between learners, which adaptive learning systems can use to tailor the learning experience to individual learners. For fact learning, cognitive models of human memory are well suited to tracing learners’ acquisition and forgetting of knowledge over time. Such models have shown great promise in controlled laboratory studies. To work in realistic educational settings, however, they need to be easy to deploy and their adaptive components should be based on individual differences relevant to the educational context and outcomes. Here, we focus on predicting university students’ exam performance using a model-based adaptive fact-learning system. The data presented here indicate that the system provides tangible benefits to students in naturalistic settings. The model’s estimate of a learner’s rate of forgetting predicts overall grades and performance on individual exam questions. This encouraging case study highlights the value of model-based adaptive fact-learning systems in classrooms.
References
Anderson, J. R. (2007). How Can the Human Mind Occur in the Physical Universe? New York: Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195324259.001.0001
Anderson, J. R., Bothell, D., Lebiere, C., & Matessa, M. (1998). An integrated theory of list memory. Journal of Memory and Language, 38(4), 341–380. https://doi.org/10.1006/jmla.1997.2553
Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: Beliefs, techniques, and illusions. Annual Review of Psychology, 64, 417–444. https://doi.org/10.1146/annurev-psych-113011-143823
Blasiman, R. N., Dunlosky, J., & Rawson, K. A. (2017). The what, how much, and when of study strategies: Comparing intended versus actual study behaviour. Memory, 25(6), 784–792. https://doi.org/10.1080/09658211.2016.1221974
Bloom, B. S. (1974). Time and learning. American Psychologist, 29(9), 682–688. https://doi.org/10.1037/h0037632
Boulton, C. A., Kent, C., & Williams, H. T. P. (2018). Virtual learning environment engagement and learning outcomes at a “bricks-and-mortar” university. Computers & Education, 126, 129–142. https://doi.org/10.1016/j.compedu.2018.06.031
Dempster, F. N. (1988). The spacing effect: A case study in the failure to apply the results of psychological research. American Psychologist, 43(8), 627–634. https://doi.org/10.1037/0003-066X.43.8.627
Dunlosky, J., & Rawson, K. A. (2015). Practice tests, spaced practice, and successive relearning: Tips for classroom use and for guiding students’ learning. Scholarship of Teaching and Learning in Psychology, 1(1), 72–78. https://doi.org/10.1037/stl0000024
Fortier, M. S., Vallerand, R. J., & Guay, F. (1995). Academic motivation and school performance: Toward a structural model. Contemporary Educational Psychology, 20(3), 257–274. https://doi.org/10.1006/ceps.1995.1017
Gelman, A., & Hill, J. (2006). Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge, UK: Cambridge University Press. https://doi.org/10.1017/CBO9780511790942
Goldstone, R. L., & Lupyan, G. (2016). Discovering psychological principles by mining naturally occurring data sets. Topics in Cognitive Science, 8(3), 548–568. https://doi.org/10.1111/tops.12212
Griffiths, T. L. (2015). Manifesto for a new (computational) cognitive revolution. Cognition, 135, 21–23. https://doi.org/10.1016/j.cognition.2014.11.026
Gurung, R. A. R., Weidert, J., & Jeske, A. (2012). Focusing on how students study. Journal of the Scholarship of Teaching and Learning, 10(1), 28–35. Retrieved from https://scholarworks.iu.edu/journals/index.php/josotl/article/view/1734
Hartwig, M. K., & Dunlosky, J. (2012). Study strategies of college students: Are self-testing and scheduling related to achievement? Psychonomic Bulletin & Review, 19(1), 126–134. https://doi.org/10.3758/s13423-011-0181-y
Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. New York: Springer Science & Business Media. https://doi.org/10.1007/978-0-387-84858-7
Karpicke, J. D., Butler, A. C., & Roediger, H. L. (2009). Metacognitive strategies in student learning: Do students practise retrieval when they study on their own? Memory, 17(4), 471–479. https://doi.org/10.1080/09658210802647009
Karpicke, J. D., & Roediger, H. L. (2008). The critical importance of retrieval for learning. Science, 319(5865), 966–968. https://doi.org/10.1126/science.1152408
Kim, A. S. N., Wiseheart, M., & Rosenbaum, R. S. (2019). The spacing effect stands up to big data. Behavior Research Methods, 51, 1485–1497. https://doi.org/10.3758/s13428-018-1184-7
Klinkenberg, S., Straatemeier, M., & Van der Maas, H. L. J. (2011). Computer adaptive practice of maths ability using a new item response model for on the fly ability and difficulty estimation. Computers & Education, 57(2), 1813–1824. https://doi.org/10.1016/j.compedu.2011.02.003
Kornell, N. (2009). Optimizing learning using flashcards: Spacing is more effective than cramming. Applied Cognitive Psychology, 23(9), 1297–1317. https://doi.org/10.1002/acp.1537
Kornell, N., & Bjork, R. A. (2007). The promise and perils of self-regulated study. Psychonomic Bulletin & Review, 14, 219–224. https://doi.org/10.3758/BF03194055
Kornell, N., & Son, L. K. (2009). Learners’ choices and beliefs about self-testing. Memory, 17(5), 493–501. https://doi.org/10.1080/09658210902832915
McAndrew, M., Morrow, C. S., Atiyeh, L., & Pierre, G. C. (2016). Dental student study strategies: Are self-testing and scheduling related to academic performance? Journal of Dental Education, 80(5), 542–552. https://doi.org/10.1002/j.0022-0337.2016.80.5.tb06114.x
Meehan, M., & McCallig, J. (2019). Effects on learning of time spent by university students attending lectures and/or watching online videos. Journal of Computer Assisted Learning, 35(2), 283–293. https://doi.org/10.1111/jcal.12329
Mettler, E., Massey, C. M., & Kellman, P. J. (2016). A comparison of adaptive and fixed schedules of practice. Journal of Experimental Psychology: General, 145(7), 897–917. https://doi.org/10.1037/xge0000170
Nijboer, M. (2011). Optimal Fact Learning: Applying Presentation Scheduling to Realistic Conditions. University of Groningen, Groningen, Netherlands. (Unpublished master’s thesis.)
Pavlik, P. I., & Anderson, J. R. (2005). Practice and forgetting effects on vocabulary memory: An activation-based model of the spacing effect. Cognitive science, 29(4), 559–586. https://doi.org/10.1207/s15516709cog0000_14
Pavlik, P. I., & Anderson, J. R. (2008). Using a model to compute the optimal schedule of practice. Journal of Experimental Psychology: Applied, 14(2), 101–117. https://doi.org/10.1037/1076-898X.14.2.101
Putnam, A. L., Sungkhasettee, V.W., & Roediger, H. L. (2016). Optimizing learning in college: Tips from cognitive psychology. Perspectives on Psychological Science, 11(5), 652–660. https://doi.org/10.1177/1745691616645770
Ridgeway, K., Mozer, M. C., & Bowles, A. R. (2017). Forgetting of foreign-language skills: A corpus-based analysis of online tutoring software. Cognitive Science, 41(4), 924–949. https://doi.org/10.1111/cogs.12385
Ritter, S., Anderson, J. R., Koedinger, K. R., & Corbett, A. (2007). Cognitive Tutor: Applied research in mathematics education. Psychonomic Bulletin & Review, 14(2), 249–255. https://doi.org/10.3758/BF03194060
Sense, F., Behrens, F., Meijer, R. R., & van Rijn, H. (2016). An individual’s rate of forgetting is stable over time but differs across materials. Topics in Cognitive Science, 8(1), 305–321. https://doi.org/10.1111/tops.12183
Sense, F., Meijer, R. R., & van Rijn, H. (2018). Exploration of the rate of forgetting as a domain-specific individual differences measure. Frontiers in Education, 3(112). https://doi.org/10.3389/feduc.2018.00112
Settles, B., & Meeder, B. (2016). A trainable spaced repetition model for language learning. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 7–12 August 2016, Berlin, Germany (pp. 1848–1858). Association for Computational Linguistics. https://doi.org/10.18653/v1/p16-1174
Shmueli, G. (2010). To explain or to predict? Statistical Science, 25(3), 289–310. https://doi.org/10.2139/ssrn.1351252
Taraban, R., Maki, W. S., & Rynearson, K. (1999). Measuring study time distributions: Implications for designing computer-based courses. Behavior Research Methods, Instruments, & Computers, 31(2), 263–269. https://doi.org/10.3758/BF03207718
Taraban, R., Rynearson, K., & Stalcup, K. A. (2001). Time as a variable in learning on the World-Wide Web. Behavior Research Methods, 33(2), 217–225. https://doi.org/10.3758/bf03195368
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1), 267–288. https://doi.org/10.1111/j.1467-9868.2011.00771.x
van den Broek, G. S. E., Takashima, A., Wiklund-H¨ornqvist, C., Karlsson Wirebring, L., Segers, E., Verhoeven, L., & Nyberg, L. (2016). Neurocognitive mechanisms of the “testing effect”: A review. Trends in Neuroscience and Education, 5(2), 52–66. https://doi.org/10.1016/j.tine.2016.05.001
VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16, 227–265. Retrieved from https://dl.acm.org/doi/10.5555/1435351.1435353
van Rijn, H., van Maanen, L., & van Woudenberg, M. (2009). Passing the test: Improving learning gains by balancing spacing and testing effects. In A. Hoses, D. Peebles, & R. Cooper (Eds.), Proceedings of the Ninth International Conference on Cognitive Modeling, 24–26 July 2009, Manchester, UK (pp. 110–115). Retrieved from https://iccm-conference.neocities.org/2009/proceedings/cd/papers/200/paper200.pdf
Wise, A. F., & Shaffer, D. W. (2015). Why theory matters more than ever in the age of big data. Journal of Learning Analytics, 2(2), 5–13. https://doi.org/10.18608/jla.2015.22.2
Wissman, K. T., Rawson, K. A., & Pyc, M. A. (2012). How and when do students use flashcards? Memory, 20(6), 568–579. https://doi.org/10.1080/09658211.2012.687052
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2021 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
