Game Learning Analytics:
Blending Visual and Data Mining Techniques to Improve Serious Games and to Better Understand Player Learning
DOI:
https://doi.org/10.18608/jla.2022.7633Keywords:
serious games, game learning analytics, game-based learning, stealth assessment, visualization, research paperAbstract
Game learning analytics (GLA) comprise the collection, analysis, and visualization of player interactions with serious games. The information gathered from these analytics can help us improve serious games and better understand player actions and strategies, as well as improve player assessment. However, the application of analytics is a complex and costly process that is not yet generalized in serious games. Using a standard data format to collect player interactions is essential: the standardization allows us to simplify and systematize every step in developing tools and processes compatible with multiple games. In this paper, we explore a combination of 1) an exploratory visualization tool that analyzes player interactions in the game and provides an overview of their actions, and 2) an assessment approach, based on the collection of interaction data for player assessment. We describe some of the different opportunities offered by analytics in game-based learning, the relevance of systematizing the process by using standards and game-independent analyses and visualizations, and the different techniques (visualizations, data mining models) that can be applied to yield meaningful information to better understand learners’ actions and results in serious games.
References
ADL. (2012). Experience API (xAPI) Standard. https://adlnet.gov/projects/xapi/
ADL. (2021, October 20). IEEE to standardize xAPI v2.0 as an international standard. https://adlnet.gov/news/2021/10/20/IEEE-to-Standardize-xAPI-v2.0-as-an-International-Standard/
Alonso-Fernández, C., Cano, A. R., Calvo-Morata, A., Freire, M., Martínez-Ortiz, I., & Fernández-Manjón, B. (2019). Lessons learned applying learning analytics to assess serious games. Computers in Human Behavior, 99, 301–309. https://doi.org/10.1016/j.chb.2019.05.036
Alonso-Fernández, C., Freire, M., Martínez-Ortiz, I., & Fernández-Manjón, B. (2021). Improving evidence-based assessment of players using serious games. Telematics and Informatics, 60, 101583. https://doi.org/10.1016/j.tele.2021.101583
Andersen, E., Liu, Y.-E., Apter, E., Boucher-Genesse, F., & Popović, Z. (2010). Gameplay analysis through state projection. Proceedings of the 5th International Conference on the Foundations of Digital Games (FDG ’10), 19–21 June, Monterey, CA, USA (pp. 1–8). ACM Press. https://doi.org/10.1145/1822348.1822349
Anderson, C. G., Dalsen, J., Kumar, V., Berland, M., & Steinkuehler, C. (2018). Failing up: How failure in a game environment promotes learning through discourse. Thinking Skills and Creativity, 30, 135–144. https://doi.org/10.1016/j.tsc.2018.03.002
Andrews-Todd, J., Mislevy, R. J., LaMar, M., & de Klerk, S. (2021). Virtual performance-based assessments. In A. A. von Davier, R. J. Mislevy, & J. Hao (Eds.), Computational psychometrics: New methodologies for a new generation of digital learning and assessment (pp. 45–60). Methodology of Educational Measurement and Assessment. Springer. https://doi.org/10.1007/978-3-030-74394-9_4
Carter, M., & Egliston, B. (2021). What are the risks of virtual reality data? Learning analytics, algorithmic bias and a fantasy of perfect data. New Media & Society, 146144482110127. https://doi.org/10.1177/14614448211012794
Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., & Boyle, J. M. (2012). A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, 59(2), 661–686. https://doi.org/10.1016/j.compedu.2012.03.004
Dörner, R., Göbel, S., Effelsberg, W., & Wiemeyer, J. (Eds.). (2016). Serious games: Foundations, concepts and practice. Springer. https://doi.org/10.1007/978-3-319-40612-1
Dreiseitl, S., & Ohno-Machado, L. (2002). Logistic regression and artificial neural network classification models: A methodology review. Journal of Biomedical Informatics, 35(5–6), 352–359. https://doi.org/10.1016/S1532-0464(03)00034-0
Eagle, M., Johnson, M., Barnes, T., & Boyce, A. (2013). Exploring player behavior with visual analytics. Proceedings of the 8th International Conference on the Foundations of Digital Games (FDG ’13), 14–17 May, Chania, Crete, Greece (pp. 380–383). Society for the Advancement of the Science of Digital Games. http://www.fdg2013.org/program/papers/short06_eagle_etal.pdf
e-UCM. (2020). T-Mon: Monitor of traces in xAPI-SG in Python. GitHub. https://github.com/e-ucm/t-mon
European Commission. (2016). What does the General Data Protection Regulation (GDPR) govern? https://ec.europa.eu/info/law/law-topic/data-protection/reform/what-does-general-data-protection-regulation-gdpr-govern_en
Freire, M., Serrano-Laguna, Á., Iglesias, B. M., Martínez-Ortiz, I., Moreno-Ger, P., & Fernández-Manjón, B. (2016). Game learning analytics: Learning analytics for serious games. In M. Spector, B. Lockee, & M. Childress (Eds.), Learning, Design, and Technology (pp. 1–29). Springer International Publishing. https://doi.org/10.1007/978-3-319-17727-4_21-1
Hao, J., & Mislevy, R. J. (2018). The evidence trace file: A data structure for virtual performance assessments informed by data analytics and evidence-centered design. ETS Research Report Series, 2018(1), 1–16. https://doi.org/10.1002/ets2.12215
Khosravi, H., Shabaninejad, S., Bakharia, A., Sadiq, S., Indulska, M., & Gašević, D. (2021). Intelligent learning analytics dashboards: Automated drill-down recommendations to support teacher data exploration. Journal of Learning Analytics, 8(3), 133–154. https://doi.org/10.18608/jla.2021.7279
Kim, Y. J., Lin, G., & Ruipérez-Valiente, J. A. (2021). Expanding teacher assessment literacy with the use of data visualizations in game-based assessment. In M. Sahin & D. Ifenthaler (Eds.), Visualizations and dashboards for learning analytics: Advances in analytics for learning and teaching (pp. 399–419). Springer. https://doi.org/10.1007/978-3-030-81222-5_18
Liu, M., Han, S., Shao, P., Cai, Y., & Pan, Z. (2021). The current landscape of research and practice on visualizations and dashboards for learning analytics. In M. Sahin & D. Ifenthaler (Eds.), Visualizations and dashboards for learning analytics: Advances in analytics for learning and teaching (pp. 23–46). Springer. https://doi.org/10.1007/978-3-030-81222-5_2
Marchiori, E. J., Ferrer, G., Fernández-Manjón, B., Povar Marco, J., Suberviola González, J. F., & Giménez Valverde, A. (2012). Instrucción en maniobras de soporte vital básico mediante videojuegos a escolares: comparación de resultados frente a un grupo control. Emergencias: Revista de la Sociedad Española de Medicina de Urgencias y Emergencias, 24(6), 433–437.
Minović, M., & Milovanović, M. (2013). Real-time learning analytics in educational games. Proceedings of the 1st International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM ’13), 14–15 November, Salamanca, Spain (pp. 245–251). ACM Press. https://doi.org/10.1145/2536536.2536574
Montavon, G., Samek, W., & Müller, K.-R. (2018). Methods for interpreting and understanding deep neural networks. Digital Signal Processing, 73, 1–15. https://doi.org/10.1016/j.dsp.2017.10.011
Nery Mendes, I., de Araújo Nogueira, M., Valente Mendes, F., Noura Teixeira, O., & Almeida dos Santos, V. (2022). The use of serious games for learning cardiopulmonary resuscitation procedures: A systematic mapping of the literature. In B. Sobota (Ed.), Computer game development. IntechOpen. https://doi.org/10.5772/intechopen.102399
Nguyen, T.-H. D., Seif El-Nasr, M., & Canossa, A. (2015). Glyph: Visualization tool for understanding problem solving strategies in puzzle games. Proceedings of the 10th International Conference on the Foundations of Digital Games (FDG ’15), 22–25 June, Pacific Grove, CA, USA (pp. 1–9). Society for the Advancement of the Science of Digital Games. https://doi.org/10.48550/arXiv.2106.13742
Pérez-Colado, V. M., Pérez-Colado, I. J., Martínez-Ortiz, I., Freire-Morán, M., & Fernández-Manjón, B. (2021). Democratizing game learning analytics for serious games. In F. de Rosa, I. Marfisi Schottman, J. Baalsrud Hauge, F. Bellotti, P. Dondio, & M. Romero (Eds.), Games and learning alliance (GALA 2021; pp. 164–173). Lecture Notes in Computer Science, vol. 13134. Springer. https://doi.org/10.1007/978-3-030-92182-8_16
Seif El-Nasr, M., Drachen, A., & Canossa, A. (Eds.). (2013). Game analytics: Maximizing the value of player data. Springer. https://doi.org/10.1007/978-1-4471-4769-5
Serrano-Laguna, Á., Martínez-Ortiz, I., Haag, J., Regan, D., Johnson, A., & Fernández-Manjón, B. (2017). Applying standards to systematize learning analytics in serious games. Computer Standards & Interfaces, 50, 116–123. https://doi.org/10.1016/j.csi.2016.09.014
Shute, V. (2011). Stealth assessment in computer-based games to support learning. In S. Tobias & J. D. Fletcher (Eds.), Computer games and instruction (pp. 503–524). IAP Information Age Publishing.
Shute, V., & Rahimi, S. (2021). Stealth assessment of creativity in a physics video game. Computers in Human Behavior, 116, 106647. https://doi.org/10.1016/j.chb.2020.106647
Shute, V., Rahimi, S., & Lu, X. (2019). Supporting learning in educational games: Promises and challenges. In P. Díaz, A. Ioannou, K. Bhagat, & J. Spector (Eds.), Learning in a digital world: Smart computing and intelligence (pp. 59–81). Springer. https://doi.org/10.1007/978-981-13-8265-9_4
Shute, V., Rahimi, S., Smith, G., Ke, F., Almond, R., Dai, C., Kuba, R., Liu, Z., Yang, X., & Sun, C. (2021). Maximizing learning without sacrificing the fun: Stealth assessment, adaptivity and learning supports in educational games. Journal of Computer Assisted Learning, 37(1), 127–141. https://doi.org/10.1111/jcal.12473
Shute, V., Smith, G., Kuba, R., Dai, C.-P., Rahimi, S., Liu, Z., & Almond, R. (2021). The design, development, and testing of learning supports for the Physics Playground game. International Journal of Artificial Intelligence in Education, 31(3), 357–379. https://doi.org/10.1007/s40593-020-00196-1
Siqueira, T. V., da Silva Garcia Nascimento, J., Gouvêa de Oliveira, J. L., da Silva Garcia Regino, D., & Barcellos Dalri, M. C. (2020). The use of serious games as an innovative educational strategy for learning cardiopulmonary resuscitation: An integrative review. Revista gaucha de enfermagem, 41, e20190293. https://doi.org/10.1590/1983-1447.2020.20190293
Syal, S., & Nietfeld, J. L. (2020). The impact of trace data and motivational self-reports in a game-based learning environment. Computers & Education, 157, 103978. https://doi.org/10.1016/j.compedu.2020.103978
Tlili, A., & Chang, M. (Eds.). (2019). Data analytics approaches in educational games and gamification systems. Springer. https://doi.org/10.1007/978-981-32-9335-9
Tlili, A., Chang, M., Moon, J., Liu, Z., Burgos, D., Chen, N. S., & Kinshuk. (2021). A systematic literature review of empirical studies on learning analytics in educational games. International Journal of Interactive Multimedia and Artificial Intelligence, 7(2), 250–261. https://doi.org/10.9781/ijimai.2021.03.003
Yang, X., Rahimi, S., Shute, V., Kuba, R., Smith, G., & Alonso-Fernández, C. (2021). The relationship among prior knowledge, accessing learning supports, learning outcomes, and game performance in educational games. Educational Technology Research and Development, 69(2), 1055–1075. https://doi.org/10.1007/s11423-021-09974-7
Yu, Z., Gao, M., & Wang, L. (2021). The effect of educational games on learning outcomes, student motivation, engagement and satisfaction. Journal of Educational Computing Research, 59(3), 522–546. https://doi.org/10.1177/0735633120969214
Zeng, J., Parks, S., & Shang, J. (2020). To learn scientifically, effectively, and enjoyably: A review of educational games. Human Behavior and Emerging Technologies, 2(2), 186–195. https://doi.org/10.1002/hbe2.188
Zhang, Q., & Rutherford, T. (2022). Grade 5 students’ elective replay after experiencing failures in learning fractions in an educational game: When does replay after failures benefit learning? Proceedings of the 12th International Conference on Learning Analytics and Knowledge (LAK ’22), 21–25 March, Online (pp. 98–106). ACM Press. https://doi.org/10.1145/3506860.3506873
Zhao, Y., Wang, Y., Zhang, J., Fu, C.-W., Xu, M., & Moritz, D. (2022). KD-box: Line-segment-based KD-tree for interactive exploration of large-scale time-series data. IEEE Transactions on Visualization and Computer Graphics, 28(1), 890–900. https://doi.org/10.1109/TVCG.2021.3114865
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
