Associations of Research Questions, Analytical Techniques, and Learning Insight in Temporal Educational Research
A Systematic Mapping Study
DOI:
https://doi.org/10.18608/jla.2023.7745Keywords:
learning analytics, systematic mapping, temporal analytics, research paperAbstract
Learning has a temporal characteristic in nature, which means that it occurs over the passage of time. The research on the temporal aspects of learning faces several challenges, one of which is utilizing appropriate analytical techniques to exploit the temporal data. There is no coherent guide to selecting certain temporal techniques to lead to results that truthfully uncover underlying phenomena. To fill this gap, this systematic mapping study contributes to understanding the type of questions and approaches in works in the area of temporal educational research. This study aims to analyze different components of published research and explores the current trends in educational studies that explicitly consider the temporal aspect. Using the thematic coding method, we identified trends in three components, including asked research questions, utilized methodological techniques, and inferred insight about learning. The distribution of codes regarding asked research questions showed that the highest number of studies focused on method development or proposing a methodological framework. We discussed that methodological development, with the underlying theory, led to identifying learning indicators that can provide the ability to identify individual students with respect to the learning concepts of interest. In terms of utilized techniques, there was a strong trend in visualization analysis and process mining. This study found that to discover insight into learning, it is important to utilize techniques that are interpretable to characterize temporal patterns.
References
Bakeman, R., & Gottman, J. M. (1997). Observing interaction: An introduction to sequential analysis (2nd ed.). Cambridge University Press. https://doi.org/10.1017/CBO9780511527685
Basit, T. (2003). Manual or electronic? The role of coding in qualitative data analysis. Educational Research, 45(2), 143–154. https://doi.org/10.1080/0013188032000133548
Beheshitha, S. S., Gašević, D., & Hatala, M. (2015). A process mining approach to linking the study of aptitude and event facets of self-regulated learning. Proceedings of the 5th International Conference on Learning Analytics and Knowledge (LAK ʼ15), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 265–269). ACM Press. https://doi.org/10.1145/2723576.2723628
Bienkowski, M., Feng, M., & Means, B. (2014). Enhancing teaching and learning through educational data mining and learning analytics: An issue brief. U.S. Department of Education. https://tech.ed.gov/wp-content/uploads/2014/03/edm-la-brief.pdf
Bogarín, A., Cerezo, R., & Romero, C. (2018). A survey on educational process mining. WIREs Data Mining and Knowledge Discovery, 8(1). https://doi.org/10.1002/widm.1230
Shirvani Boroujeni, M., & Dillenbourg, P. (2019). Discovery and temporal analysis of MOOC study patterns. Journal of Learning Analytics, 6(1), 16–33. https://doi.org/10.18608/jla.2019.61.2
Chen, B., Knight, S., & Wise, A. F. (2018). Critical issues in designing and implementing temporal analytics. Journal of Learning Analytics, 5(1), 1–9. https://doi.org/10.18608/jla.2018.53.1
Chen, B., Resendes, M., Chai, C. S., & Hong, H.-Y. (2017). Two tales of time: Uncovering the significance of sequential patterns among contribution types in knowledge-building discourse. Interactive Learning Environments, 25(2), 162–175. https://doi.org/10.1080/10494820.2016.1276081
Chen, B., Wise, A. F., Knight, S., & Cheng, B. H. (2016). Putting temporal analytics into practice: The 5th international workshop on temporality in learning data. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼ16), 25–29 April 2016, Edinburgh, UK (pp. 488–489). ACM Press. https://doi.org/10.1145/2883851.2883865
de Barba, P. G., Malekian, D., Oliveira, E. A., Bailey, J., Ryan, T., & Kennedy, G. (2020). The importance and meaning of session behaviour in a MOOC. Computers & Education, 146, 103772. https://doi.org/10.1016/j.compedu.2019.103772
Dickersin, K., Scherer, R., & Lefebvre, C. (1994). Systematic reviews: Identifying relevant studies for systematic reviews. BMJ, 309(6964), 1286–1291. https://doi.org/10.1136/bmj.309.6964.1286
Du, X., Zhang, M., Shelton, B. E., & Hung, J.-L. (2022). Learning anytime, anywhere: A spatio-temporal analysis for online learning. Interactive Learning Environments, 30(1), 34–48. https://doi.org/10.1080/10494820.2019.1633546
Fan, Y., Saint, J., Singh, S., Jovanović, J., & Gašević, D. (2021). A learning analytic approach to unveiling self-regulatory processes in learning tactics. Proceedings of the 11th International Conference on Learning Analytics and Knowledge (LAK ’21), 12–16 April 2021, Irvine, CA, USA (pp. 184–195). ACM Press. https://doi.org/10.1145/3448139.3448211
Gabadinho, A., Ritschard, G., Müller, N. S., & Studer, M. (2011). Analyzing and visualizing state sequences in R with TraMineR. Journal of Statistical Software, 40(4). https://doi.org/10.18637/jss.v040.i04
Gašević, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59, 64–71. https://doi.org/10.1007/s11528-014-0822-x
Gašević, D., Kovanović, V., & Joksimović, S. (2017). Piecing the learning analytics puzzle: A consolidated model of a field of research and practice. Learning: Research and Practice, 3(1), 63–78. https://doi.org/10.1080/23735082.2017.1286142
Google Scholar. (n.d.). Educational Technology – Google scholar metric. https://scholar.google.ca/citations?view_op=top_venues&hl=en&vq=eng_educationaltechnology
Hatala, M., Nazeri, S., & Salehian Kia, F. (2023). Progression of students’ SRL processes in subsequent programming problem-solving tasks and its association with tasks outcomes. The Internet and Higher Education, 56, 100881. https://doi.org/10.1016/j.iheduc.2022.100881
Huang, L., & Lajoie, S. P. (2021). Process analysis of teachers’ self-regulated learning patterns in technological pedagogical content knowledge development. Computers & Education, 166, 104169. https://doi.org/10.1016/j.compedu.2021.104169
Jovanović, J., Dawson, S., Joksimović, S., & Siemens, G. (2020). Supporting actionable intelligence: Reframing the analysis of observed study strategies. Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK ’20), 23–27 March 2020, Frankfurt, Germany (pp. 161–170). ACM Press. https://doi.org/10.1145/3375462.3375474
Jovanović, J., Gašević, D., Dawson, S., Pardo, A., & Mirriahi, N. (2017). Learning analytics to unveil learning strategies in a flipped classroom. The Internet and Higher Education, 33, 74–85. https://doi.org/10.1016/j.iheduc.2017.02.001
Kinnebrew, J. S., Segedy, J. R., & Biswas, G. (2014). Analyzing the temporal evolution of students’ behaviors in open-ended learning environments. Metacognition and Learning, 9, 187–215. https://doi.org/10.1007/s11409-014-9112-4
Kitchenham, B. A., Budgen, D., & Pearl Brereton, O. (2011). Using mapping studies as the basis for further research: A participant-observer case study. Information and Software Technology, 53(6), 638–651. https://doi.org/10.1016/j.infsof.2010.12.011
Knight, S., Wise, A. F., & Chen., B. (2017). Time for change: Why learning analytics needs temporal analysis. Journal of Learning Analytics, 4(3), 7–17. http://doi.org/10.18608/jla.2017.43.2
Lee, A. V. Y., & Tan, S. C. (2017). Temporal analytics with discourse analysis: Tracing ideas and impact on communal discourse. Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 120–127). ACM Press. https://doi.org/10.1145/3027385.3027386
Liu, S., Kang, L., Liu, Z., Fang, J., Yang, Z., Sun, J., Wang, M., & Hu, M. (2021). Computer-supported collaborative concept mapping: The impact of students’ perceptions of collaboration on their knowledge understanding and behavioral patterns. Interactive Learning Environments, 1–20. https://doi.org/10.1080/10494820.2021.1927115
Mahzoon, M. J., Maher, M. L., Eltayeby, O., Dou, W., & Grace, K. (2018). A sequence data model for analyzing temporal patterns of student data. Journal of Learning Analytics, 5(1), 55–74. https://doi.org/10.18608/jla.2018.51.5
Matcha, W., Gašević, D., Ahmad Uzir, N., Jovanović, J., Pardo, A., Maldonado-Mahauad, J., & Pérez-Sanagustín, M. (2019). Detection of learning strategies: A comparison of process, sequence and network analytic approaches. Proceedings of the 14th European Conference on Technology Enhanced Learning (EC-TEL 2019), 16–19 September 2019, Delft, The Netherlands (Lecture Notes in Computer Science, vol. 11722, pp. 525–540). Springer. https://doi.org/10.1007/978-3-030-29736-7_39
Mohabbati, B., Asadi, M., Gašević, D., Hatala, M., & Müller, H. A. (2013). Combining service-orientation and software product line engineering: A systematic mapping study. Information and Software Technology, 55(11), 1845–1859. https://doi.org/10.1016/j.infsof.2013.05.006
Molenaar, I. (2014). Advances in temporal analysis in learning and instruction. Frontline Learning Research, 2(4), 15–24. https://doi.org/10.14786/flr.v2i4.118
Nazeri, S., Hatala, M., & Salehian Kia, F. (2023). When to intervene? Utilizing two facets of temporality in students’ SRL processes in a programming course. Proceedings of the 13th International Conference on Learning Analytics and Knowledge (LAK ’22), 13–17 March 2023, Arlington, TX, USA (pp. 293–302). ACM Press. https://doi.org/10.1145/3576050.3576095
Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods, 16(1). https://doi.org/10.1177/1609406917733847
Petersen, K., Feldt, R., Mujtaba, S., & Mattsson, M. (2008). Systematic mapping studies in software engineering. Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering (EASE 2008), 26–27 June 2008, Bari, Italy (pp. 1–10). BCS Learning and Development. https://doi.org/10.14236/ewic/ease2008.8
Petersen, K., Vakkalanka, S., & Kuzniarz, L. (2015). Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology, 64, 1–18. https://doi.org/10.1016/j.infsof.2015.03.007
Reimann, P. (2009). Time is precious: Variable- and event-centred approaches to process analysis in CSCL research. International Journal of Computer-Supported Collaborative Learning, 4, 239–257. https://doi.org/10.1007/s11412-009-9070-z
Reimann, P., Markauskaite, L., & Bannert, M. (2014). e-Research and learning theory: What do sequence and process mining methods contribute? British Journal of Educational Technology, 45(3), 528–540. https://doi.org/10.1111/bjet.12146
Riel, J., Lawless, K. A., & Brown, S. W. (2018). Timing matters: Approaches for measuring and visualizing behaviours of timing and spacing of work in self-paced online teacher professional development courses. Journal of Learning Analytics, 5(1), 25–40. https://doi.org/10.18608/jla.2018.51.3
Saint, J., Fan, Y., Singh, S., Gašević, D., & Pardo, A. (2021). Using process mining to analyse self-regulated learning: A systematic analysis of four algorithms. Proceedings of the 11th International Conference on Learning Analytics and Knowledge (LAK ’21), 12–16 April 2021, Irvine, CA, USA (pp. 333–343). ACM Press. https://doi.org/10.1145/3448139.3448171
Sher, N., Kent, C., & Rafaeli, S. (2020). How “networked” are online collaborative concept-maps? Introducing metrics for quantifying and comparing the “networkedness” of collaboratively constructed content. Education Sciences, 10(10), 267. https://doi.org/10.3390/educsci10100267
Wang, M., Guo, W., Le, H., & Qiao, B. (2020). Reply to which post? An analysis of peer reviews in a high school SPOC. Interactive Learning Environments, 28(5), 574–585. https://doi.org/10.1080/10494820.2019.1696840
Wang, Q., Saha, K., Gregori, E., Joyner, D., & Goel, A. (2021). Towards mutual theory of mind in human-AI interaction: How language reflects what students perceive about a virtual teaching assistant. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI ’21), 8–13 May 2021, Yokohama, Japan (pp. 1–14). ACM Press. https://doi.org/10.1145/3411764.3445645
Wise, A. F., & Shaffer, D. W. (2015). Why theory matters more than ever in the age of big data. Journal of Learning Analytics, 2(2), 5–13. https://doi.org/10.18608/jla.2015.22.2
Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: An overview. Educational Psychologist, 25(1), 3–17. https://doi.org/10.1207/s15326985ep2501_2
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
