Amplifying Student and Administrator Perspectives on Equity and Bias in Learning Analytics
Alone Together in Higher Education
DOI:
https://doi.org/10.18608/jla.2023.7775Keywords:
stakeholder persepctives, bias, equity, learning analytics, higher education, research paperAbstract
When higher education institutions (HEIs) have the potential to collect large amounts of learner data, it is important to consider the spectrum of stakeholders involved with and impacted by the use of learning analytics. This qualitative research study aims to understand the degree of concern with issues of bias and equity in the uses of learner data as perceived by students, diversity and inclusion leaders, and senior administrative leaders in HEIs. An interview study was designed to investigate stakeholder voices that generate, collect, and utilize learning analytics from eight HEIs in the United States. A phased inductive coding analysis revealed similarities and differences in the three stakeholder groups regarding concerns about bias and equity in the uses of learner data. The study findings suggest that stakeholders have varying degrees of data literacy, thus creating conditions of inequality and bias in learning data. By centring the values of these critical stakeholder groups and acknowledging that intersections and hierarchies of power are critical to authentic inclusion, this study provides additional insight into proactive measures that institutions could take to improve equity, transparency, and accountability in their responsible learning analytics efforts.
References
Ahn, J., Campos, F., Hays, M., & DiGiacomo, D. (2019). Designing in context: Reaching beyond usability in learning analytics dashboard design. Journal of Learning Analytics, 6(2), 70–85. http://dx.doi.org/10.18608/jla.2019.62.5
Bethune, A., McCambly, H., Rodgers, A. J., Villanosa, K., & Nzinga, K. (2020). Insights on (in)equity initiatives in the context of discourse, organizations, and identity. Proceedings of the 14th International Conference of the Learning Sciences (ICLS ’20), 19–23 June 2020, Nashville, TN, USA (pp. 468–475). International Society of the Learning Sciences. https://repository.isls.org/bitstream/1/6674/1/468-475.pdf
Broughan, C., & Prinsloo, P. (2020). (Re)centering students in learning analytics: In conversation with Paulo Freire. Assessment & Evaluation in Higher Education, 45(4), 617–628. https://dx.doi.org/10.1080/02602938.2019.1679716
Brown, J. T. (2017). The seven silos of accountability in higher education: Systematizing multiple logics and fields. Research & Practice in Assessment, 11, 41–58. https://files.eric.ed.gov/fulltext/EJ1137933.pdf
Buckingham Shum, S., Ferguson, R., & Martinez-Maldonado, R. (2019). Human-centred learning analytics. Journal of Learning Analytics, 6(2), 1–9. https://dx.doi.org/10.18608/jla.2019.62.1
Chen, B., & Zhu, H. (2019). Towards value-sensitive learning analytics design. Proceedings of the 9th International Conference on Learning Analytics and Knowledge (LAK ’19), 4–8 March 2019, Tempe, AZ, USA (pp. 343–352). ACM Press. https://dx.doi.org/10.1145/3303772.3303798
Colvin, C., Rogers, T., Wade, A., Dawson, S., Gašević, D., Buckingham Shum, S., Nelson, K., Alexander, S., Lockyer, L., Kennedy, G., Corrin, L., Fisher, J. (2016). Student retention and learning analytics: A snapshot of Australian practices and a framework for advancement. Australian Office for Learning and Teaching.
Conde, M. Á., & Hernández-García, Á. (2019). Learning analytics: The end of the beginning. Proceedings of the 7th International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM ’19), 16–18 October 2019, León, Spain (pp. 248–252). ACM Press. https://dx.doi.org/10.1145/3362789.3362943
Danaher, M., Cook, J., Danaher, G., Coombes, P., & Danaher, P. A. (2013). Situating education research with marginalized communities. In Researching Education with Marginalized Communities, 1–22. Palgrave Macmillan. https://dx.doi.org/10.1057/9781137012685_1
Dollinger, M., Liu, D., Arthars, N., & Lodge, J. (2019). Working together in learning analytics towards the co-creation of value. Journal of Learning Analytics, 6(2), 10–26. http://dx.doi.org/10.18608/jla.2019.62.2
Doughty, H. A. (2014). Surveillance, big data analytics and the death of privacy. The College Quarterly, 17(3). https://files.eric.ed.gov/fulltext/EJ1049869.pdf
Drachsler, H., & Greller, W. (2016). Privacy and analytics: It’s a DELICATE issue — a checklist for trusted learning analytics. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼ16), 25–29 April 2016, Edinburgh, UK (pp. 89–98). ACM Press. https://dx.doi.org/10.1145/2883851.2883893
Esmonde, I. (2016). Power and privilege in the learning sciences: Critical and socio-cultural theories of learning. In I. Esmonde & A. N. Booker (Eds.), Power and privilege in the learning sciences: Critical and socio-cultural theories of learning (pp. 6–27). Taylor & Francis Group.
Ferguson, R. (2013). Learning analytics: Drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5–6), 304–317. https://dx.doi.org/10.1504/IJTEL.2012.051816
Ferguson, R., & Buckingham Shum, S. (2012). Social learning analytics: Five approaches. In S. Buckingham Shum, D. Gašević, & R. Ferguson (Eds.), Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK ʼ12), 29 April–2 May 2012, Vancouver, BC, Canada (pp. 23–33). ACM Press. https://dx.doi.org/10.1145/2330601.2330616
Ferguson, R. (2019). Ethical challenges for learning analytics. Journal of Learning Analytics, 6(3), 25–30. https://doi.org/10.18608/jla.2019.63.5
Friedman, B., & Hendry, D. G. (2019). Value sensitive design: Shaping technology with moral imagination. MIT Press. http://dx.doi.org/10.7551/mitpress/7585.001.0001
Funes, M., & Mackness, J. (2018). When inclusion excludes: A counter narrative of open online education. Learning, Media and Technology, 43, 119–138. http://dx.doi.org/10.1080/17439884.2018.1444638
Fynn, A., Adamiak, J., & Young, K. (2022). A Global South perspective on learning analytics in an open distance e-learning (ODeL) institution. In P. Prinsloo, S. Slade, & M. Khalil (Eds.), Learning analytics in open and distributed learning (pp. 31–45). Springer. https://doi.org/10.1007/978-981-19-0786-9_3
Gee, J. P., & Handford, M. (Eds.). (2013). The Routledge handbook of discourse analysis. Routledge.
Gourlay, L. (2015). Open education as a “heterotopia of desire.” Learning, Media and Technology, 40(3), 310–327, https://dx.doi.org/10.1080/17439884.2015.1029941
Green, B. (2021). Data science as political action: Grounding data science in a politics of justice. Journal of Social Computing, 2(3), 249–265. https://dx.doi.org/10.23919/JSC.2021.0029
Holstein, K., & Doroudi, S. (2019). Fairness and equity in learning analytics systems (FairLAK). Companion Proceedings of the 9th International Conference on Learning Analytics and Knowledge (LAK ’19), 23–27 March 2020, Frankfurt, Germany (pp. 1–2). Society for Learning Analytics Research (SoLAR).
Hon, L., & Brunner, B. (2002). Measuring public relationships among students and administrators at the University of Florida. Journal of Communication Management, 6(3), 227–238. https://doi.org/10.1108/13632540210807071
Hope, A. (2015.) Foucault’s toolbox: Critical insights for education and technology researchers. Learning, Media, and Technology, 40(4), 536–549. https://dx.doi.org/10.1080/17439884.2014.953546.
Ifenthaler, D., & Schumacher, C. (2016). Student perceptions of privacy principles for learning analytics. Educational Technology Research and Development, 64, 923–938. https://dx.doi.org/10.1007/s11423-016-9477-y
Jones, K. M. L., Asher, A., Goben, A., Perry, M. R., Salo, D., Briney, K. A., & Robertshaw, M. B. (2020). “We’re being tracked at all times”: Student perspectives of their privacy in relation to learning analytics in higher education. Journal of the Association for Information Science and Technology, 71(9), 1044–1059. https://dx.doi.org/10.1002/asi.24358
Lang, C., Macfadyen, L. P., Slade, S., Prinsloo, P., & Sclater, N. (2018). The complexities of developing a personal code of ethics for learning analytics practitioners: Implications for institutions and the field. Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK ’18), 5–9 March 2018, Sydney, NSW, Australia (pp. 436–440). ACM Press. https://dx.doi.org/10.1145/3170358.3170396
Li, W., Sun, K., Schaub, F., & Brooks, C. (2022). Disparities in students’ propensity to consent to learning analytics. International Journal of Artificial Intelligence in Education, 32, 564–608. https://dx.doi.org/10.1007/s40593-021-00254-2
Luedke, C. L. (2017). Person first, student second: Staff and administrators of color supporting students of color authentically in higher education. Journal of College Student Development, 58(1), 37–52. https://dx.doi.org/10.1353/csd.2017.0002
Macfadyen, L. P., & Dawson, S. (2012). Numbers are not enough: Why e-learning analytics failed to inform an institutional strategic plan. Journal of Educational Technology & Society, 15(3), 149–163. http://www.jstor.org/stable/jeductechsoci.15.3.149
Perrotta, C., & Williamson, B. (2018). The social life of learning analytics: Cluster analysis and the “performance” of algorithmic education. Learning, Media and Technology, 43(1), 3–16. https://dx.doi.org/10.1080/17439884.2016.1182927
Philip, T. M., Bang, M., & Jackson, K. (2018). Articulating the “how,” the “for what,” the “for whom,” and the “with whom” in concert: A call to broaden the benchmarks of our scholarship. Cognition and Instruction, 36(2), 83–88. https://doi.org/10.1080/07370008.2018.1413530
Prinsloo, P. (2018). Context matters: An African perspective on institutionalizing learning analytics. Include us all! Directions for adoption of learning analytics in the Global South, 24–35.
Prinsloo, P. (2020). Of “black boxes” and algorithmic decision-making in (higher) education: A commentary. Big Data & Society, 7(1). https://dx.doi.org/10.1177/2053951720933994
Prinsloo, P., Slade, S., & Khalil, M. (2018). Stuck in the middle? Making sense of the impact of micro, meso and macro institutional, structural and organisational factors on implementing learning analytics. Proceedings of the Conference of the European Distance and E-Learning Network (EDEN 2018), 17–20 June 2018, Genova, Italy (pp. 342–350). http://www.eden-online.org/wp-content/uploads/2018/06/Annual_2018_Genova_Proceedings_ISSN.pdf
Raffaghelli, J. E. (2020). Is data literacy a catalyst of social justice? A response from nine data literacy initiatives in higher education. Education Sciences, 10(9), 233. https://doi.org/10.3390/educsci10090233
Raffaghelli, J. E., Manca, S., Stewart, B., Prinsloo, P., & Sangrà, A. (2020). Supporting the development of critical data literacies in higher education: Building blocks for fair data cultures in society. International Journal of Educational Technology in Higher Education, 17, 58. https://doi.org/10.1186/s41239-020-00235-w
Roberts, L. D., Chang, V., & Gibson, D. (2016). Ethical considerations in adopting a university- and system-wide approach to data and learning analytics. In B. K. Daniel (Ed.), Big data and learning analytics in higher education (pp. 89–108). Springer. http://dx.doi.org/10.1007/978-3-319-06520-5_7
Rogers, R., Schaenen, I., Schott, C., O’Brien, K., Trigos-Carrillo, L., Starkey, K., & Chasteen, C. C. (2016). Critical discourse analysis in education: A review of the literature, 2004 to 2012. Review of Educational Research, 86(4), 1192–1226. https://doi.org/10.3102/0034654316628993
Ropers-Huilman, B., Carwile, L., & Barnett, K. (2005). Student activists’ characterizations of administrators in higher education: Perceptions of power in “the system.” The Review of Higher Education, 28(3), 295–312. https://doi.org/10.1353/rhe.2005.0012
Rubel, A., & Jones, K. M. L. (2016). Student privacy in learning analytics: An information ethics perspective. The Information Society, 32(2), 143–159. https://dx.doi.org/10.1080/01972243.2016.1130502
Saldaña, J. (2021). The coding manual for qualitative researchers (4th ed.). Sage.
Scheffel, M., Tsai, Y.-S., Gašević, D., & Drachsler, H. (2019). Policy matters: Expert recommendations for learning analytics policy. In M. Scheffel, J. Broisin, V. Pammer-Schindler, A. Ioannou, & J. Schneider (Eds.), Transforming learning with meaningful technologies (pp. 510–524). Springer. https://dx.doi.org/10.1007/978-3-030-29736-7_38
Selwyn, N. (2015). The discursive construction of education in the digital age. In R. H. Jones, A. Chik, & C. Hafner (Eds.), Discourse and digital practices: Doing discourse analysis in the digital age (pp. 238–252). https://doi.org/10.4324/9781315726465
Siemens, G. (2013). Learning analytics: The emergence of a discipline. American Behavioral Scientist, 57(10), 1380–1400. https://dx.doi.org/10.1177/0002764213498851
Slade, S., & Prinsloo, P. (2013). Learning analytics: Ethical issues and dilemmas. American Behavioral Scientist, 57(10), 1510–1529. https://dx.doi.org/10.1177/0002764213479366
Subotzky, G., & Prinsloo, P. (2011). Turning the tide: A socio-critical model and framework for improving student success in open distance learning at the University of South Africa. Distance Education, 32(2), 177–193. https://doi.org/10.1080/01587919.2011.584846
Swenson, J. (2014). Establishing an ethical literacy for learning analytics. Proceedings of the 4th International Conference on Learning Analytics and Knowledge (LAK ʼ14), 24–28 March 2014, Indianapolis, IN, USA (pp. 246–250). ACM Press. https://dx.doi.org/10.1145/2567574.2567613
Tsai, Y.-S., & Gašević, D. (2017). Learning analytics in higher education — challenges and policies: A review of eight learning analytics policies. Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 233–242). ACM Press. https://dx.doi.org/10.1145/3027385.3027400
Tsai, Y.-S., Perrotta, C., & Gašević, D. (2020). Empowering learners with personalised learning approaches? Agency, equity and transparency in the context of learning analytics. Assessment & Evaluation in Higher Education, 45(4), 554–567. https://dx.doi.org/10.1080/02602938.2019.1676396
Tzimas, D., & Demetriadis, S. (2021). Ethical issues in learning analytics: A review of the field. Educational Technology Research and Development, 69, 1101–1133. https://doi.org/10.1007/s11423-021-09977-4
Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in Human Behavior, 89, 98–110. https://doi.org/10.1016/j.chb.2018.07.027
Wasson, B., Hansen, C., & Netteland, G. (2016). Data literacy and use for learning when using learning analytics for learners. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼ16), 25–29 April 2016, Edinburgh, UK. CEUR Workshop Proceedings, Vol. 1596, paper #6. http://ceur-ws.org/Vol-1596/paper6.pdf
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
