Supporting Decision-Making for Promoting Teaching and Learning Innovation
A Multiple Case Study
DOI:
https://doi.org/10.18608/jla.2024.8131Keywords:
decision-making in higher education, teaching and learning centres, dashboards, data-driven decisions, learning analytics, analytics tools, research paperAbstract
The quality of the data and the amount of correct information available is key to informed decision-making. Higher education institutions (HEIs) often employ various decision support systems (DSSs) to make better choices. However, there is a lack of systems to assist with decision-making to promote innovation in teaching and learning. In this study, we evaluate an analytic tool called PROF-XXI that supports strategic decision-making of teaching and learning centres (TLCs) by identifying their competencies in teaching and learning innovation. Through a multiple case study conducted with three Latin American universities and supported by quantitative and qualitative data, we observed how this tool is used and how it facilitates strategic decision-making. Our findings indicate that the tool is accessible, user-friendly, and effective in 1) initiating identification and systematic reflection of institutional competency levels in teaching and learning innovation, 2) enhancing understanding of strengths and weaknesses as well as identifying opportunities for innovation, 3) supporting TLCs with short- and long-term decision-making, and 4) continuously evaluating their strategies, programs, and initiatives. This research can benefit policymakers in higher education who are involved in measuring institutional competencies to improve teaching quality or in making strategic decisions related to teaching and learning innovation.
References
Albon, S. P., Iqbal, I., & Pearson, M. L. (2016). Strategic planning in an educational development centre: Motivation, management, and messiness. Collected Essays on Learning and Teaching, 9, 207–226. https://doi.org/10.22329/celt.v9i0.4427
Almeida, F., & Amoedo, N. (2018). Decision support system for internship management in higher education. International Journal of Information Systems and Social Change, 9(1), 40–57. https://doi.org/10.4018/IJISSC.2018010103
Atkins, B., Koroluk, J., & Stranach, M. (2017). Canadian teaching and learning centres on Facebook and Twitter: An exploration through social media. TechTrends, 61(3), 253–262. https://doi.org/10.1007/s11528-016-0144-2
Bresfelean, V. P., & Ghisoiu, N. (2010). Higher education decision making and decision support systems. WSEAS Transactions on Advances in Engineering Education, 7(2), 43–52. http://www.wseas.us/e-library/transactions/education/2010/89-327.pdf
Bugwandeen, K., & Ungerer, M. (2019). Exploring the design of performance dashboards in relation to achieving organisational strategic goals. South African Journal of Industrial Engineering, 30(2), 161–175. https://doi.org/10.7166/30-2-2021
Catino, M. (2013). Organizational myopia: Problems of rationality and foresight in organizations. Cambridge University Press. https://doi.org/10.1017/CBO9781139208819
Chanwijit, J., Lomwongpaiboon, W., Dowjam, O., & Tangworakitthaworn, P. (2016). Decision support system for targeting higher education. In J. L. Mitrpanont (Ed.), Proceedings of the 2016 Fifth ICT International Student Project Conference (ICT-ISPC), 27–28 May 2016, Nakhon Pathom, Thailand (pp. 154–157). IEEE. https://doi.org/10.1109/ICT-ISPC.2016.7519259
Chick, R. C., Clifton, G. T., Peace, K. M., Propper, B. W., Hale, D. F., Alseidi, A. A., & Vreeland, T. J. (2020). Using technology to maintain the education of residents during the COVID-19 pandemic. Journal of Surgical Education, 77(4), 729–732. https://www.doi.org/10.1016/j.jsurg.2020.03.018
Cohen, L., Manion, L., & Morrison, K. (2017). Research methods in education (8th ed.). Routledge. https://doi.org/10.4324/9781315456539
Creswell, J. W. (2013). Qualitative inquiry and research design: Choosing among five approaches (3rd edition). SAGE Publications, Inc.
Cubukcu Cerasi, C., Akturk, C., & Talan, T. (2022). Categorization of decision support systems in the field of education. Proceedings of the 5th International Conference on Research in Social Sciences and Humanities (ICRSH), 10–12 June 2022, Vienna, Austria. Diamond Scientific Publishing. https://doi.org/10.33422/5th.icrsh.2022.06.10
De Laet, T. (2023). Learning dashboards for academic advising in practice. In O. Viberg & Å. Grönlund (Eds.), Practicable learning analytics (pp. 55–75). Springer, Cham. https://doi.org/10.1007/978-3-031-27646-0_4
Drew, M. R., Falcone, B., & Baccus, W. L. (2018). What does the system usability scale (SUS) measure? Validation using think aloud verbalization and behavioral metrics. In A. Marcus & W. Wang (Eds.), Design, User Experience, and Usability: Theory and Practice: Proceedings of the 7th International Conference of Design, User Experience, and Usability (DUXU 2018), 15–20 July 2018, Las Vegas, NV, USA (pp. 356–366). Springer. https://doi.org/10.1007/978-3-319-91797-9_25
Eisenhardt, K. M. (1989). Building theories from case study research. The Academy of Management Review, 14(4), 532–550. https://doi.org/10.2307/258557
Fakeeh, K. (2015). Decision support systems (DSS) in higher education system. International Journal of Applied Information Systems (IJAIS), 9(2), 32–40.
Firth-Cozens, J. (2001). Multidisciplinary teamwork: The good, bad, and everything in between. BMJ Quality & Safety, 10(2), 65–66. https://doi.org/10.1136/qhc.10.2.65
Galletta, A. (2013). Mastering the semi-structured interview and beyond: From research design to analysis and publication. New York University Press. https://doi.org/10.18574/nyu/9780814732939.001.0001
Gao, M., Kortum, P., & Oswald, F. (2020). Multi-language toolkit for the system usability scale. International Journal of Human–Computer Interaction, 36(20), 1883–1901. https://doi.org/10.1080/10447318.2020.1801173
García-Morales, V. J., Garrido-Moreno, A., & Martín-Rojas, R. (2021). The transformation of higher education after the COVID disruption: Emerging challenges in an online learning scenario. Frontiers in Psychology, 12, 616059. https://doi.org/10.3389/fpsyg.2021.616059
Gray, K., & Radloff, A. (2006). Quality management of academic development work: Implementation issues and challenges. International Journal for Academic Development, 11(2), 79–90. https://doi.org/10.1080/13601440600924397
Intergovernmental Panel on Climate Change (IPCC). (2014). Foundations for decision-making. In Climate change 2014 — Impacts, adaptation and vulnerability: Part A: Global and sectoral aspects (pp. 195–228). Cambridge University Press. https://doi.org/10.1017/CBO9781107415379.007
Kadoić, N., Ređep, N. B., & Divjak, B. (2018). A new method for strategic decision-making in higher education. Central European Journal of Operations Research, 26(3), 611–628. https://doi.org/10.1007/s10100-017-0497-4
Kasap, S., Abbas, D., Khajah, M., & Ashknani, M. (2020). Developing a knowledge-driven decision support system for university/college selection problem in Kuwait. International Journal of Information and Education Technology, 10(1), 20–25. https://doi.org/10.18178/ijiet.2020.10.1.1333
Kloos, C. D., Alario-Hoyos, C., Morales, M., Rocael, H. R., Jerez, Ó., Pérez-Sanagustín, M., Kotorov, I., Fernández, S. A. R., Oliva-Córdova, L. M., Solarte, M., Jaramillo, D., Teixeira, A. M., & López, A. H. G. (2021). PROF-XXI: Teaching and learning centers to support the 21st century professor. Proceedings of the 2021 World Engineering Education Forum/Global Engineering Deans Council (WEEF/GEDC), 15–18 November 2021, Madrid, Spain (pp. 447–454). IEEE. https://doi.org/10.1109/WEEF/GEDC53299.2021.9657301
Kotorov, I., Krasylnykova, Y., Demel, J., & Blaschke, P. (2023). The effect of the Covid-19 pandemic on economic growth and R&D spending in the EU countries. In P. Vrabcová (Ed.), Proceedings of the 16th International Conference Liberec Economic Forum 2023 (LEF 2023), 5–6 September 2023, Liberec, Czechia (pp. 68–76). Technical University of Liberec. https://doi.org/10.15240/tul/009/lef-2023-08
Kotorov, I., Krasylnykova, Y., Zhdanov, P., Mazzara, M., Aslam, H., Akhmetgaraeva, E., Naumcheva, M., & Brown, J. A. (2021). Institutional commitment and leadership as prerequisites for successful comprehensive internationalization. In G. Succi, P. Ciancarini, & A. Kruglov (Eds.), Frontiers in Software Engineering: Proceedings of the First International Conference on Frontiers in Software Engineering (ICFSE 2021), 17–18 June 2021, Innopolis, Russia (pp. 1–11). Springer, Cham. https://doi.org/10.1007/978-3-030-93135-3_1
Kotorov, I., Pérez-Sanagustín, M., Mansilla, F., Krasylnykova, Y., Hadaou, F. T., & Broisin, J. (2022). Supporting the monitoring of institutional competency in learning innovation: The PROF-XXI tool. Proceedings of the 17th Latin American Conference on Learning Technologies (LACLO), 17–21 October 2022, Armenia, Colombia. IEEE. https://doi.org/10.1109/LACLO56648.2022.10013323
Krasylnykova, Y., Kotorov, I., Demel, J., Mazzara, M., & Bobrov, E. (2023). Innopolis University: An agile and resilient academic institution navigating the rocky waters of the COVID-19 pandemic. In G. Jezic, J. Chen-Burger, M. Kusek, R. Sperka, R. J. Howlett, & L. C. Jain (Eds.), Agents and multi-agent systems: Technologies and applications 2023: Proceedings of the 17th KES International Conference (KES-AMSTA 2023), 14–16 June 2023, Rome, Italy (pp. 383–392). Springer. https://doi.org/10.1007/978-981-99-3068-5_36
Malkawi, N. M. M. (2018). How to improve decision making process through decision support systems & business intelligence: Evidence from Jordan University Hospital. Journal of Economic & Management Perspectives, 12(2), 255–265.
Mazzara, M., Zhdanov, P., Bahrami, M. R., Aslam, H., Kotorov, I., Imam, M., Salem, H., Brown, J. A., & Pletnev, R. (2022). Education after COVID-19. In R. J. Howlett, L. C. Jain, J. R. Littlewood, & M. M. Balas (Eds.), Smart and sustainable technology for resilient cities and communities (pp. 193–207). Springer Singapore. https://doi.org/10.1007/978-981-16-9101-0_14
Moya, B., Turra, H., & Chalmers, D. (2019). Developing and implementing a robust and flexible framework for the evaluation and impact of educational development in higher education in Chile. International Journal for Academic Development, 24(2), 163–177. https://doi.org/10.1080/1360144X.2018.1555757
Nieto, Y., Gacía-Díaz, V., Montenegro, C., González, C. C., & González Crespo, R. (2019a). Usage of machine learning for strategic decision making at higher educational institutions. IEEE Access, 7, 75007–75017. https://doi.org/10.1109/ACCESS.2019.2919343
Nieto, Y., García-Díaz, V., Montenegro, C., & González Crespo, R. (2019b). Supporting academic decision making at higher educational institutions using machine learning-based algorithms. Soft Computing, 23(12), 4145–4153. https://doi.org/10.1007/s00500-018-3064-6
Oliva-Córdova, L. M., Garcia-Cabot, A., & Amado-Salvatierra, H. R. (2021). Learning analytics to support teaching skills: A systematic literature review. IEEE Access, 9, 58351–58363. https://doi.org/10.1109/ACCESS.2021.3070294
Palmer, S., Holt, D., & Challis, D. (2010). Australian teaching and learning centres through the eyes of their directors: Characteristics, capacities and constraints. Journal of Higher Education Policy and Management, 32(2), 159–172. https://doi.org/10.1080/13600800903575454
Pérez-Sanagustín, M., Kotorov, I., Teixeira, A., Mansilla, F., Broisin, J., Alario-Hoyos, C., Jerez, Ó., Teixeira Pinto, M. d. C., García, B., Delgado Kloos, C., Morales, M., Solarte, M., Oliva-Córdova, L. M., & Gonzalez Lopez, A. H. (2022). A competency framework for teaching and learning innovation centers for the 21st century: Anticipating the post-COVID-19 age. Electronics, 11(3), 413. https://doi.org/10.3390/electronics11030413
Ploywattanawong, L. (2016). Decision support systems model for admission to higher education. Proceedings of the 2016 2nd IEEE International Conference on Computer and Communications (ICCC), 14–17 October 2016, Chengdu, China (pp. 1359–1363). IEEE. https://doi.org/10.1109/CompComm.2016.7924926
PROF-XXI. (n.d.). Objectives. http://www.profxxi.org/?page_id=367&lang=en
Quadri, A. T., & Shukor, N. A. (2021). The benefits of learning analytics to higher education institutions: A scoping review. International Journal of Emerging Technologies in Learning (iJET), 16(23), 4–15. https://doi.org/10.3991/ijet.v16i23.27471
Rippen, M. (2005). Decision support systems (version 2). In K. James and T. Grant (Eds.), Proceedings of the 4th Australian Life Cycle Assessment Conference: Sustainability Measures for Decision Support, 23–25 February 2005, Sydney, NSW, Australia [published on CD]. Australian Life Cycle Assessment Society. https://www.vgls.vic.gov.au/client/en_AU/vgls/search/results?qu=Australian+Life+Cycle+Assessment+Conference+%284th+%3A+2005+%3A+Novotel%2C+Sydney%2C+N.S.W.%29
Sevilla-Gonzalez, M. D. R., Moreno Loaeza, L., Lazaro-Carrera, L. S., Bourguet Ramirez, B., Vázquez Rodríguez, A., Peralta-Pedrero, M. L., & Almeda-Valdes, P. (2020). Spanish version of the system usability scale for the assessment of electronic tools: Development and validation. JMIR Human Factors, 7(4), e21161. https://doi.org/10.2196/21161
Shah, M. (2014). Impact of management information systems (MIS) on school administration: What the literature says. Procedia: Social and Behavioral Sciences, 116, 2799–2804. https://doi.org/10.1016/j.sbspro.2014.01.659
Shalabi, R. (2020). The importance and applications of decision support systems (DSS) in higher education. figshare. https://doi.org/10.6084/m9.figshare.12465599.v1
Yin, R. K. (2003). Case study research: Design and methods (3rd ed.). SAGE Publications, Inc.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
