A Novel Deep Learning Model for Student Performance Prediction Using Engagement Data
DOI:
https://doi.org/10.18608/jla.2024.7985Keywords:
learning analytics, educational data mining, predictive models, deep representation learning, research paperAbstract
Technology-enhanced learning supported by virtual learning environments (VLEs) facilitates tutors and students. VLE platforms contain a wealth of information that can be used to mine insight regarding students’ learning behaviour and relationships between behaviour and academic performance, as well as to model data-driven decision-making. This study introduces a system that we termed ASIST: a novel Attention-aware convolutional Stacked BiLSTM network for student representation learning to predict their performance. ASIST exploits student academic registry, VLE click stream, and midterm continuous assessment information for their behaviour representation learning. ASIST jointly learns the student representation using five behaviour vectors. It processes the four sequential behaviour vectors using a separate stacked bidirectional long short term memory (LSTM) network. A deep convolutional neural network models the diurnal weekly interaction behaviour. It also employs the attention mechanism to assign weight to features based on their importance. Next, five encoded feature vectors are concatenated with the assessment information, and, finally, a softmax layer predicts the high-performer (H), moderate-performer (M), and at-risk (F) categories of students. We evaluate ASIST over three datasets from an Irish university, considering five evaluation metrics. ASIST achieves an area under the curve (AUC) score of 0.86 to 0.90 over the three datasets. It outperforms three baseline deep learning models and four traditional classification models. We also found that the attention mechanism has a slight impact on ASIST’s performance. The ablation analysis reveals that weekly event count has the greatest impact on ASIST, whereas diurnal weekly interaction has the least impact. The early prediction using the first seven weeks of data achieves an AUC of 0.83 up to 0.89 over the three datasets. In yearly analysis, ASIST performs best over the 2018/19 dataset and worst over the 2020/21 dataset.
References
Abulaish, M., Kumari, N., Fazil, M., & Singh, B. (2019). A graph-theoretic embedding-based approach for rumor detection in Twitter. In P. Barnaghi, G. Gottlob, Y. Manolopoulos, T. Tzouramanis, & A. Vakali (Eds.), Proceedings of the 2019 IEEE/WIC/ACM International Conference on Web Intelligence (WI 2019), 13–17 October 2019, Thessaloniki, Greece (pp. 466–470). ACM. https://doi.org/10.1145/3350546.3352569
Alam, M. M., Mohiuddin, K., Das, A. K., Islam, M. K., Kaonain, M. S., & Ali, M. H. (2018). A reduced feature based neural network approach to classify the category of students. In Proceedings of the Second International Conference on Innovation in Artificial Intelligence (ICIAI 2018), 9–12 April 2018, Shanghai, China (pp. 28–32). ACM. https://doi.org/10.1145/3194206.3194218
Ang, K. L.-M., Ge, F. L., & Seng, K. P. (2020). Big educational data and analytics: Survey, architecture and challenges. IEEE Access, 8(7), 116392–116414. https://doi.org/10.1109/ACCESS.2020.2994561
Arbel, N. (2018). How LSTM networks solve the problem of vanishing gradients (tech. rep.). Medium. https://medium.datadriveninvestor.com/how-do-lstm-networks-solve-the-problem-of-vanishing-gradients-a6784971a577
Bahdanau, D., Cho, K., & Bengio, Y. (2015). Neural machine translation by jointly learning to align and translate. In Proceedings of the Third International Conference on Learning Representations (ICLR 2015), 7–9 May 2015, San Diego, California, USA (pp. 1–15). ICLR. https://arxiv.org/abs/1409.0473
Christian, T. M., & Ayub, M. (2014). Exploration of classification using NBTree for predicting students’ performance. In Proceedings of the 2014 International Conference on Data and Software Engineering (ICODSE 2014), 26–27 November 2014, Bandung, Indonesia (pp. 1–6). IEEE Computer Society. https://doi.org/10.1109/ICODSE.2014.7062654
Chui, K. T., Fung, D. C. L., Lytras, M. D., & Lam, T. M. (2020). Predicting at-risk university students in a virtual learning environment via a machine learning algorithm. Computers in Human Behavior, 107(2), 1–7. https://doi.org/10.1016/j.chb.2018.06.032
Dowah, H. A., Al-Samarraie, H., & Fauzy,W. M. (2019). Educational data mining and learning analytics for 21st century higher education: A review and synthesis. Telematics and Informatics, 37(1), 13–49. https://doi.org/10.1016/j.tele.2019.01.007
Essa, A. (2019). Is data dark? Lessons from Borges’s “Funes the Memorius.” Journal of Learning Analytics, 6(3), 35–42.https://doi.org/10.18608/jla.2019.63.7
Fazil, M., & Abulaish, M. (2018). A hybrid approach for detecting automated spammers in twitter. IEEE Transactions on Information Forensics and Security, 13(11), 2707–2719. https://doi.org/10.1109/TIFS.2018.2825958
Fazil, M., Sah, A. K., & Abulaish, M. (2021). DeepSBD: A deep neural network model with attention mechanism for socialbot detection. IEEE Transactions on Information Forensics and Security, 16(8), 4211–4223. https://doi.org/10.1109/TIFS.2021.3102498
Gasevic, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrend, 59, 64–71. https://doi.org/10.1007/s11528-014-0822-x
Hai-tao, P., Ming-qu, F., Hong-bin, Z., Bi-zhen, Y., Jin-jiao, L., Chun-fang, L., Yan-ze, Z., & Rui, S. (2021). Predicting academic performance of students in Chinese-foreign cooperation in running schools with graph convolutional network. Neural Computing and Applications, 33(1), 637–645. https://doi.org/10.1007/s00521-020-05045-9
Hernandez-Blanco, A., Herrera-Flores, B., Tomas, D., & Navarro-Colorado, B. (2019). A systematic review of deep learning approaches to educational data mining. Complexity, 2019(1), 1–22. https://doi.org/10.1155/2019/1306039
Hidalgo, A. C., Ger, P. M., & Valentin, L. D. L. F. (2022). Using meta-learning to predict student performance in virtual learning environments. Applied Intelligence, 52(7), 3352–3365. https://doi.org/10.1007/s10489-021-02613-x
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
Hu, Q., & Rangwala, H. (2019). Reliable deep grade prediction with uncertainty estimation. In Proceedings of the Ninth International Conference on Learning Analytics and Knowledge (LAK 2019), 4–8 March 2019, Tempe, Arizona, USA (pp. 76–85). ACM. https://doi.org/10.1145/3303772.3303802
Huang, S., & Fang, N. (2010). Prediction of student academic performance in an engineering dynamics course: Development and validation of multivariate regression models. International Journal of Engineering Education, 26(4), 1008–1017.https://www.ijee.ie/contents/c260410.html
Ihianle, I. K., Nwajana, A. O., Ebenuwa, S., Otuka, R. I., Owa, K., & Orisatoki, M. O. (2020). A deep learning approach for human activities recognition from multimodal sensing devices. IEEE Access, 8(1), 179028–179038. https://doi.org/10.1109/ACCESS.2020.3027979
Jeslet, S., Komarasamy, D., & Hermina, J. J. (2021). Student result prediction in Covid-19 lockdown using machine learning techniques. In Proceedings of the International Conference on Innovative Technology for Sustainable Development (ICITSD-2021), 27–29 January 2021, Chennai, India (pp. 1–9). IOP Publisher. https://doi.org/10.1088/1742-6596/1911/1/012008
Karimi, H., Derr, T., Huang, J., & Tang, J. (2020). Online academic course performance prediction using relational graph convolutional neural network. In A. N. Rafferty, J. Whitehill, V. Cavalli-Sforza, & C. Romero (Eds.), Proceedings of the 13th International Conference on Educational Data Mining (EDM 2020), 10–13 July 2020, online (pp. 444–450). ACM.
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(1), 436–444. https://doi.org/10.1038/nature14539
Li, X., Zhang, Y., Cheng, H., Li, M., & Yin, B. (2022). Student achievement prediction using deep neural network from multi-source campus data. Complex & Intelligent Systems, 8, 5143–5156. https://doi.org/10.1007/s40747-022-00731-8
Long, P., & Siemens, G. (2011). Penetrating the fog: Analytics in learning and education. Educause Review, 46(5), 31–40.https://er.educause.edu/articles/2011/9/penetrating-the-fog-analytics-in-learning-and-education
Marbouti, F., Diefes-Dux, H. A., & Madhavan, K. (2016). Models for early prediction of at-risk students in a course using standards-based grading. Computers & Education, 103(9), 1–15. https://doi.org/10.1016/j.compedu.2016.09.005
Migueis, V., Freitas, A., Garcia, P. J., & Silv, A. (2018). Early segmentation of students according to their academic performance:A predictive modelling approach. Decision Support Systems, 115(9), 36–51. https://doi.org/10.1016/j.dss.2018.09.001
Mishra, T., Kumar, D., & Gupta, S. (2014). Mining students’ data for performance prediction. In Proceedings of the Fourth International Conference on Advanced Computing & Communication Technologies (ACCT 2014), 8–9 February 2014, Rohtak, Haryana, India (pp. 255–262). IEEE Computer Society. https://doi.org/10.1109/ACCT.2014.105
Mubarak, A. A., Cao, H., & Hezam, I. M. (2021). Deep analytic model for student dropout prediction in massive open online courses. Computers and Electrical Engineering, 93(6), 1–14. https://doi.org/10.1016/j.compeleceng.2021.107271
Palacios, C. A., Reyes-Suarez, J. A., Bearzotti, L. A., Leiva, V., & Marchant, C. (2021). Knowledge discovery for higher education student retention based on data mining: Machine learning algorithms and case study in Chile. Entropy, 23(4), 1–23. https://doi.org/10.3390/e23040485
Pascanu, R., Gulcehre, C., Cho, K., & Bengio, Y. (2014). How to construct deep recurrent neural networks. In Proceedings of the International Conference on Learning Representations (ICLR 2014), 14–16 April 2014, Banff, Alberta, Canada (pp. 1–13). Arxiv. https://arxiv.org/abs/1312.6026
Priya, S., Ankit, T., & Divyansh, D. (2021). Student performance prediction using machine learning. In D. Hemanth, M.Elhosney, T. Nguyen, & S. Lakshmanan (Eds.), Proceedings of the Virtual International Conference on Advances in Parallel Computing Technologies and Applications (ICAPTA 2021), 15–16 April 2021, online (pp. 167–174). IOS Press. https://doi.org/10.3233/APC210137
Qiu, L., Liu, Y., Hu, Q., & Liu, Y. (2018). Student dropout prediction in massive open online courses by convolutional neural networks. Soft Computing, 23(10), 10287–10301. https://doi.org/10.1007/s00500-018-3581-3
Raga, R. C., & Raga, J. D. (2019). Early prediction of student performance in blended learning courses using deep neural networks. In Proceedings of the Fifth International Symposium on Educational Technology (ISET 2019), 2–4 July 2019, Hradec KrÅLalovÅLe, Czechia (pp. 39–43). IEEE. https://doi.org/10.1109/ISET.2019.00018
Ramanathan, K., & Thangavel, B. (2021). Minkowski Sommon Feature Map-based Densely Connected Deep Convolution Network with LSTM for academic performance prediction. Concurrency and Computation: Practice and Experience, 33(13), 1–17. https://doi.org/10.1002/cpe.6244
Rizvi, S., Rientiesa, B., & Khoja, S. A. (2019). The role of demographics in online learning; a decision tree based approach. Computers & Education, 137(4), 32–47. https://doi.org/10.1016/j.compedu.2019.04.001
Romero, C., Espejo, P. G., Zafra, A., Romero, J. R., & Ventura, S. (2013). Web usage mining for predicting final marks of students that use Moodle courses. Computer Applications in Engineering Education, 21(1), 135–146. https://doi.org/10.1002/cae.20456
Roy, P. K., Tripathy, A. K., Das, T. K., & Gao, X. (2020). A framework for hate speech detection using deep convolutional neural network. IEEE Access, 8(11), 204951–204962. https://doi.org/10.1109/ACCESS.2020.3037073
Santos, J. L., Klerkx, J., Duval, E., Gago, D., & Rodriguez, L. (2014). Success, activity and drop-outs in MOOCs an exploratory study on the UNED COMA courses. Proceedings of the Fourth International Conference on Learning Analytics and Knowledge (LAK 2014), 24–28 March 2014, Indianapolis, Indiana, USA, 98–102. https://doi.org/10.1145/2567574.2567627
Selwyn, N. (2020). Re-imagining “Learning Analytics” . . . a case for starting again? The Internet and Higher Education, 46(7), 1–9. https://doi.org/10.1016/j.iheduc.2020.100745
Shafiq, D. A. K., Marjani, M., Habeeb, R. A. A., & Asirvatham, D. (2022). Student retention using educational data mining and predictive analytics: A systematic literature review. IEEE Access, 10(7), 72480–72503. https://doi.org/10.1109/ACCESS.2022.3188767
Sharada, N., Shashi, M., & Xiong, X. (2018). Modeling student knowledge retention using deep learning and random forests. Journal of Engineering and Applied Sciences, 13(6), 1347–1353. https://doi.org/10.3923/jeasci.2018.1347.1353
Strecht, P., Cruz, L., Soares, C., Mendes-Moreira, J., & Abreu, R. (2015). A comparative study of classification and regression algorithms for modelling students’ academic performance. In O. Santos, J. Boticario, C. Romero, M. Pechenizkiy, A. Merceron, P. Mitros, J. Luna, C. Mihaescu, P. Moreno, A. Hershkovitz, S. Ventura, & M. Desmarais (Eds.), Proceedings of the International Conference on Educational Data Mining (EDM 2015), 26–29 June 2015, Madrid, Spain (pp. 1–4). International Educational Data Mining Society. https://www.educationaldatamining.org/EDM2015/proceedings/short392-395.pdf
Waheed, H., Hassan, S.-U., Aljohani, N. R., Hardman, J., Alelyani, S., & Nawaz, R. (2020). Predicting academic performance of students from VLE big data using deep learning models. Computers in Human Behavior, 104(1), 1–13. https://doi.org/10.1016/j.chb.2019.106189
Waheed, H., Hassan, S.-U., Nawaz, R., Aljohani, N. R., Chend, G., & Gasevic, D. (2022). Early prediction of learners at risk in self-paced education: A neural network approach. Expert Systems with Applications, 213(9), 1–13. https://doi.org/10.1016/j.eswa.2022.118868
Walsh, J. N., & Risquez, A. (2020). Using cluster analysis to explore the engagement with a flipped classroom of native and non-native English-speaking management students. The International Journal of Management Education, 18(2), 1–9. https://doi.org/10.1016/j.ijme.2020.100381
Wang,W., Yu, H., & Miao, C. (2017). Deep model for dropout prediction in MOOCs. In Proceedings of the Second International Conference on Crowd Science and Engineering (ICCSE 2017), 6–9 July 2017, Beijing, China (pp. 26–32). ACM. https://doi.org/10.1145/3126973.3126990
Wasif, M., Waheed, H., Aljohani, N. R., & Hassan, S.-U. (2019). Understanding student learning behavior and predicting their performance. Cognitive Computing in Technology-Enhanced Learning, 1(1), 1–28. https://doi.org/10.4018/978-1-5225-9031-6.ch001
Xing, W., & Du, D. (2018). Dropout prediction in MOOCs: Using deep learning for personalized intervention. Journal of Educational Computing, 57(3), 547–570. https://doi.org/10.1177/0735633118757015
Yagci, M. (2022). Educational data mining: Prediction of students’ academic performance using machine learning algorithms.Smart Learning Environments, 9(11), 1–19. https://doi.org/10.1186/s40561-022-00192-z
Yin, W., Kann, K., Yu, M., & Schutze, H. (2017). Comparative study of CNN and RNN for natural language processing. arXiv:1702.01923v1. https://doi.org/10.48550/arXiv.1702.01923
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2018). Recent trends in deep learning based natural language processing. IEEE Computational Intelligence Magazine, 13(3), 55–75. https://doi.org/10.1109/MCI.2018.2840738
Zhang, L., Qi, G.-J., Wang, L., & Luo, J. (2019). AET vs. AED: Unsupervised representation learning by auto-encoding transformations rather than data. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019), 15–20 June 2019, Long Beach, California, USA (pp. 2542–2550). IEEE/CVF. https://doi.org/10.1109/CVPR.2019.00265
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
