Supporting Student Agency with a Student-Facing Learning Analytics Dashboard
Perceptions of an Interdisciplinary Development Team
DOI:
https://doi.org/10.18608/jla.2023.7729Keywords:
interdisciplinary development work, student agency, student-facing dashboard, ethics, research paperAbstract
Learning analytics dashboard (LAD) development has been criticized for being too data-driven and for developers lacking an understanding of the nontechnical aspects of learning analytics (LA). The ability of developers to address their understanding of learners as well as systematic efforts to involve students in the development process are central to creating pedagogically grounded student-facing dashboards. However, limited research is available about developer perceptions on supporting students with LA. We examined an interdisciplinary LA development team’s (IDT) perceptions of and intentions to support student agency, and the student-facing LAD development process. Qualitative content analysis supported by a social cognitive theory framework was conducted on interviews (N = 12) to analyze the IDT’s perceptions of student agency. IDT members had differing conceptions of student agency but agreed that it manifests in strategic study progression and planning, as well as in active interpretation and use of LA-based feedback. IDT members had differing views on student involvement in the LAD development process. Communication challenges within an IDT and limited resources were mentioned, impeding development work. The results of this study highlight the importance of fostering communication among IDT members about guiding pedagogical design principles and the systematic use of educational concepts in LA development processes.
References
Abras, C., Maloney-Krichmar, D., & Preece, J. (2004). User-centered design. In W. Bainbridge (Ed.), Encyclopedia of human–computer interaction (pp. 445–456). Sage Publications.
Ahn, J., Campos, F., Hays, M., & Digiacomo, D. (2019). Designing in context: Reaching beyond usability in learning analytics dashboard design. Journal of Learning Analytics, 6(2), 70–85. https://doi.org/10.18608/jla.2019.62.5
Altman, D. G. (1990). Practical statistics for medical research. Chapman & Hall.
Bandura, A. (1977). Social learning theory. Prentice-Hall.
Bandura, A. (1982). Self-efficacy mechanism in human agency. American Psychologist, 37(2), 122–147. https://doi.org/10.1037/0003-066X.37.2.122
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall.
Bandura, A. (1997). Self-efficacy: The exercise of control. W. H. Freeman & Company.
Bandura, A. (2001). Social cognitive theory: An agentic perspective. Annual Review of Psychology, 52, 1–26. https://doi.org/10.1146/annurev.psych.52.1.1
Bandura, A. (2002). Growing primacy of human agency in adaptation and change in the electronic era. European Psychologist, 7(1), 2–16. https://doi.org/10.1027/1016-9040.7.1.2
Bennett, L., & Folley, S. (2020). Four design principles for learner dashboards that support student agency and empowerment. Journal of Applied Research in Higher Education, 12(1), 15–26. https://doi.org/10.1108/JARHE-11-2018-0251
Bergvall-Kåreborn, B., & Ståhlbrost, A. (2008). Participatory design: One step back or two steps forward? Proceedings of the 10th Anniversary Conference on Participatory Design 2008 (PDC ’08) 1–4 October 2008, Bloomington, IN, USA (pp. 102–111). AMC Press. https://www.researchgate.net/publication/221631236_Participatory_design_one_step_back_or_two_steps_forward
Birch, H., & Demmans Epp, C. (2015). Participatory design with music students: Empowering children to develop instructional technology. Proceedings of the American Educational Research Association Annual Conference (AERA 2015), 16–20 April 2015, Chicago, IL, USA. https://www.researchgate.net/publication/303785260_Participatory_design_with_music_students_Empowering_children_to_develop_instructional_technology#fullTextFileContent
Bodily, R., & Verbert, K. (2017). Review of research on student-facing learning analytics dashboards and educational recommender systems. IEEE Transactions on Learning Technologies, 10(4), 405–418. https://doi.org/10.1109/TLT.2017.2740172
Brusilovsky, P., Somyürek, S., Guerra, J., Hosseini, R., & Zadorozhny, V. (2015). The value of social: Comparing open student modeling and open social student modeling. In F. Ricci, K. Bontcheva, O. Conlan, & S. Lawless (Eds.), Proceedings of the 23rd International Conference on User Modeling, Adaptation and Personalization (UMAP 2015), 29 June–3 July 2015, Dublin, Ireland (pp. 44–55). Springer. https://doi.org/10.1007/978-3-319-20267-9_4
Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. Journal of the Learning Sciences, 6(3), 271–315. https://doi.org/10.1207/s15327809jls0603_1
Dawe, M. (2007). “Let me show you what I want”: Engaging individuals with cognitive disabilities and their families in design. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ʼ07), 28 April–3 May 2007, San Jose, CA (pp. 2177–2182). ACM Press. https://doi.org/10.1145/1240866.1240976
de Quincey, E., Turner, M., Williams, N., & Kyriacou, T. (2016). Learner analytics: The need for user-centred design in learning analytics. AMSYS, 3(9). https://doi.org/10.4108/eai.23-8-2016.151643
Derry, S. J., Schunn, C. D., & Gernsbacher, M. A. (2005). Interdisciplinary collaboration: An emerging cognitive science. Lawrence Erlbaum Associates.
Dollinger, M., Liu, D., Arthars, N., & Lodge, J. (2019). Working together in learning analytics toward the co-creation of value. Journal of Learning Analytics, 6(2), 10–26. http://dx.doi.org/10.18608/jla.2019.62.2
Eteläpelto, A., Vähäsantanen, K., Hökkä, P., & Paloniemi, S. (2013). What is agency? Conceptualizing professional agency at work. Educational Research Review, 10, 45–65. https://doi.org/10.1016/j.edurev.2013.05.001
Ferguson, R. (2012). Learning analytics: Drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5–6), 304–317. https://doi.org/10.1504/IJTEL.2012.051816
Ferguson, R. (2019). Ethical challenges for learning analytics. Journal of Learning Analytics, 6(3), 25–30. https://doi.org/10.18608/jla.2019.63.5
Fleiss, J. L., Cohen, J., & Everitt, B. S. (1969). Large sample standard errors of kappa and weighted kappa. Psychological Bulletin, 72(5), 323–327. https://doi.org/10.1037/h0028106
Gardiner, P. (2020). Learning to think together: Creativity, interdisciplinary collaboration and epistemic control. Thinking Skills and Creativity, 38, 100749. https://doi.org/10.1016/j.tsc.2020.100749
Gutiérrez, F., Seipp, K., Ochoa, X., Chiluiza, K., De Laet, T., & Verbert, K. (2020). LADA: A learning analytics dashboard for academic advising. Computers in Human Behavior, 107, 105826. https://doi.org/10.1016/j.chb.2018.12.004
Howell, J. A., Roberts, L. D., Seaman, K., & Gibson, D. C. (2018). Are we on our way to becoming a “helicopter university”? Academics’ views on learning analytics. Technology, Knowledge and Learning, 23, 1–20. https://doi.org/10.1007/s10758-017-9329-9
Jääskelä, P., Heilala, V., Kärkkäinen, T., & Häkkinen, P. (2021). Student agency analytics: Learning analytics as a tool for analysing student agency in higher education. Behaviour & Information Technology, 40(8), 790–808. https://doi.org/10.1080/0144929X.2020.1725130
Jivet, I., Scheffel, M., Drachsler, H., & Specht, M. (2017). Awareness is not enough: Pitfalls of learning analytics dashboards in the educational practice. Proceedings of the 12th European Conference on Technology Enhanced Learning (EC-TEL 2017), 12–15 September 2017, Tallinn, Estonia (pp. 82–96). Lecture Notes in Computer Science, Springer. https://doi.org/10.1007/978-3-319-66610-5_7
Johanes, P., & Thille, C. (2019). The heart of educational data infrastructures = Conscious humanity and scientific responsibility, not infinite data and limitless experimentation. British Journal of Educational Technology, 50(6), 2959–2973. https://doi.org/10.1111/bjet.12862
Khosravi, H., Gyamfi, G., Hanna, B. E., Lodge, J., & Abdi, S. (2021). Bridging the gap between theory and empirical research in evaluative judgment. Journal of Learning Analytics, 8(3), 117–132. https://doi.org/10.18608/jla.2021.7206
Klašnja-Milićević, A., & Ivanović, M. (2018). Learning analytics: New flavor and benefits for educational environments. Informatics in Education, 17(2), 285–300. https://doi.org/10.15388/infedu.2018.15
Klein, C., Lester, J., Rangwala, H., & Johri, A. (2019). Technological barriers and incentives to learning analytics adoption in higher education: Insights from users. Journal of Computing in Higher Education, 31, 604–625. https://doi.org/10.1007/s12528-019-09210-5
Klemenčič, M. (2017). From student engagement to student agency: Conceptual considerations of European policies on student-centered learning in higher education. Higher Education Policy, 30, 69–85. https://doi.org/10.1057/s41307-016-0034-4
Knight, S., Wise, A. F., & Chen, B. (2017). Time for change: Why learning analytics needs temporal analysis. Journal of Learning Analytics, 4(3), 7–17. https://doi.org/10.18608/jla.2017.43.2
Li, I., Dey, A., & Forlizzi, J. (2010). A stage-based model of personal informatics systems. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ʼ10), 10–15 April 2010, Atlanta, GA, USA (pp. 557–566). ACM Press. https://doi.org/10.1145/1753326.1753409
Lodge, J. M., Alhadad, S. S. J., Lewis, M. J., & Gašević, D. (2017). Inferring learning from big data: The importance of a transdisciplinary and multidimensional approach. Technology, Knowledge and Learning, 22, 385–400. https://doi.org/10.1007/s10758-017-9330-3
Long, P., & Siemens, G. (2011). Penetrating the fog: Analytics in learning and education. EDUCAUSE Review, 46(5), 31–40.
Lumivero. (2020). NVivo. https://lumivero.com/products/nvivo/
Markauskaite, L., & Goodyear, P. (2017). Epistemic fluency and professional education: Innovation, knowledgeable action, and actionable knowledge. Springer Dordrecht. https://doi.org/10.1007/978-94-007-4369-4
Martin, J. (2004). Self-regulated learning, social cognitive theory, and agency. Educational Psychologist, 39(2), 135–145. https://doi.org/10.1207/s15326985ep3902_4
Mittelmeier, J., Edwards, R. L., Davis, S. K., Nguyen, Q., Murphy, V. L., Brummer, L., & Rienties, B. (2018). “A double-edged sword. This is powerful but it could be used destructively”: Perspectives of early career education researchers on learning analytics. Frontline Learning Research, 6(2), 20–38. https://doi.org/10.14786/flr.v6i2.348
OECD. (2018). The future of education and skills: Education 2030. https://www.oecd.org/education/2030/oecd-education-2030-position-paper.pdf
Pardo, A., & Siemens, G. (2014). Ethical and privacy principles for learning analytics. British Journal of Educational Technology, 45(3), 438–450. https://doi.org/10.1111/bjet.12152
Pelletier, K., Brown, M., Brooks, D. C., McCormack, M., Reeves, J., Arbino, N., Bozkurt, A., Crawford, S., Czerniewicz, L., Gibson, R., Linder, K., Mason, J., & Mondelli, V. (2021). 2021 EDUCAUSE horizon report: Teaching and learning edition. EDUCAUSE. https://www.educause.edu/horizon-report-teaching-and-learning-2021
Prieto, L. P., Rodríguez-Triana, M. J., Martínez-Maldonado, R., Dimitriadis, Y., & Gašević, D. (2019). Orchestrating learning analytics (OrLA): Supporting inter-stakeholder communication about adoption of learning analytics at the classroom level. Australasian Journal of Educational Technology, 35(4), 14–33. https://doi.org/10.14742/ajet.4314
Prinsloo, P., & Slade, S. (2014). Educational triage in open distance learning: Walking a moral tightrope. The International Review of Research in Open and Distributed Learning, 15(4), 306–331. https://doi.org/10.19173/irrodl.v15i4.1881
Prinsloo, P., & Slade, S. (2016). Student vulnerability, agency and learning analytics: An exploration. Journal of Learning Analytics, 3(1), 159–182. http://dx.doi.org/10.18608/jla.2016.31.10
Punch, K. F., & Oancea, A. (2014). Introduction to research methods in education (2nd ed.). SAGE Publications.
Roberts, L. D., Howell, J. A., & Seaman, K. (2017). Give me a customizable dashboard: Personalized learning analytics dashboards in higher education. Technology, Knowledge and Learning, 22, 317–333. https://doi.org/10.1007/s10758-017-9316-1
Roberts, L. D., Howell, J. A., Seaman, K., & Gibson, D. C. (2016). Student attitudes toward learning analytics in higher education: “The Fitbit version of the learning world.” Frontiers in Psychology, 7. https://doi.org/10.3389/fpsyg.2016.01959
Sarmiento, J. P., & Wise, A. F. (2022). Participatory and co-design of learning analytics: An initial review of the literature. Proceedings of the 12th International Conference on Learning Analytics and Knowledge (LAK ’22), 21–25 March 2022, Online (pp. 535–541). ACM Press. https://doi.org/10.1145/3506860.3506910
Scholes, V. (2016). The ethics of using learning analytics to categorize students on risk. Educational Technology Research and Development, 64, 939–955. https://doi.org/10.1007/s11423-016-9458-1
Siemens, G. (2013). Learning analytics: The emergence of a discipline. American Behavioral Scientist, 57(10), 1380–1400. https://doi.org/10.1177/0002764213498851
Silvola, A., Näykki, P., Kaveri, A., & Muukkonen, H. (2021). Expectations for supporting student engagement with learning analytics: An academic path perspective. Computers & Education, 168, 104192. https://doi.org/10.1016/j.compedu.2021.104192
Slade, S., & Prinsloo, P. (2013). Learning analytics: Ethical issues and dilemmas. American Behavioral Scientist, 57(10), 1510–1529. http://dx.doi.org/10.1177/0002764213479366
Tsai, Y.-S., Perrotta, C., & Gašević, D. (2020). Empowering learners with personalised learning approaches? Agency, equity and transparency in the context of learning analytics. Assessment & Evaluation in Higher Education, 45(4), 554–567. https://doi.org/10.1080/02602938.2019.1676396
Tsai, Y.-S., Rates, D., Moreno-Marcos, P. M., Muñoz-Merino, P. J., Jivet, I., Scheffel, M., Drachsler, H., Delgado Kloos, C., & Gašević, D. (2020). Learning analytics in European higher education: Trends and barriers. Computers & Education, 155, 103993. https://doi.org/10.1016/j.compedu.2020.103933
Veiga, F. H. (2016). Assessing student engagement in school: Development and validation of a four-dimensional scale. Procedia: Social and Behavioral Sciences, 217, 813–819. https://doi.org/10.1016/j.sbspro.2016.02.153
Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in Human Behavior, 89, 98–110. https://doi.org/10.1016/j.chb.2018.07.027
West, D., Huijser, H., & Heath, D. (2016). Putting an ethical lens on learning analytics. Educational Technology Research and Development, 64, 903–922. https://doi.org/10.1007/s11423-016-9464-3
Wise, A. F., Vytasek, J., Hausknecht, S., & Zhao, Y. (2016). Developing learning analytics design knowledge in the “middle space”: The student tuning model and align design framework for learning analytics use. Online Learning Journal, 20(2). https://doi.org/10.24059/olj.v20i2.783
Zimmerman, B. J. (1995). Self-efficacy and educational development. In A. Bandura (Ed.), Self-efficacy in changing societies (pp. 202–231). Cambridge University Press. https://doi.org/10.1017/CBO9780511527692.009
Zimmerman, B. J., & Schunk, D. H. (2011). Self-regulated learning and performance. In B. J. Zimmerman & D. H. Schunk (Eds.), Handbook of self-regulation of learning and performance (pp. 1–12). Routledge.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
