Mapping the Ethics of Learning Analytics in Higher Education: A Systematic Literature Review of Empirical Research

Teresa Cerratto Pargman; Cormac McGrath

doi:10.18608/jla.2021.1

Authors

Teresa Cerratto Pargman Stockholm University https://orcid.org/0000-0001-6389-0467
Cormac McGrath Stockholm University https://orcid.org/0000-0002-8215-3646

DOI:

https://doi.org/10.18608/jla.2021.1

Keywords:

learning analytics, ethics, empirical studies, higher education, data-driven practices, systematic review

Abstract

Ethics is a prominent topic in learning analytics that has been commented on from conceptual viewpoints. For a broad range of emerging technologies, systematic literature reviews have proven fruitful by pinpointing research directions, knowledge gaps, and future research work guidance. With these outcomes in mind, we conducted a systematic literature review of the research on ethical issues that have been empirically approached in the learning analytics literature. In our final analysis, 21 articles published in the period 2014–2019 met our inclusion criteria. By analyzing this data, we seek to contribute to the field of learning analytics by 1) characterizing the type of empirical research that has been conducted on ethics in learning analytics in the context of higher education, 2) identifying the main ethical areas addressed in the selected literature, and 3) pinpointing knowledge gaps.

References

Adejo, O., & Connolly, T. (2017). Learning analytics in a shared-network educational environment: Ethical issues and countermeasures. International Journal of Advanced Computer Science and Applications, 8(4), 22–29. https://doi.org/10.14569/IJACSA.2017.080404

Arnold, K. E., & Sclater, N. (2017). Student perceptions of their privacy in learning analytics applications. Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 66–69). New York, NY: ACM. https://doi.org/10.1145/3027385.3027392

Avella, J. T., Kebritchi, M., Nunn, S. G., & Kanai, T. (2016). Learning analytics methods, benefits, and challenges in higher education: A systematic literature review. Online Learning, 20(2), 13–29. Retrieved from https://olj.onlinelearningconsortium.org/index.php/olj/article/view/790

Brooker, A., Corrin, L., Fisher, J., & Mirriahi, N. (2017). Defining “data” in conversations with students about the ethical use of learning analytics. In H. Partridge, K. Davis & J. Thomas (Eds.), Me, Us, IT! Proceedings of the 34th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE 2017), 4–6 December 2017, Toowoomba, Queensland, Australia (pp. 34–38). Australasian Society for Computers in Learning in Tertiary Education. Retrieved from https://researchbank.swinburne.edu.au/file/afa72926-26ef-4d49-8739-d1734b12fabd/1/2017-brooker-defining_data_in.pdf

Buckingham Shum, S. (2012). Learning analytics policy brief. UNESCO Institute for Information Technologies in Education. Retrieved from https://iite.unesco.org/pics/publications/en/files/3214711.pdf

Buckingham Shum, S. (2018). Transitioning education’s knowledge infrastructure: Shaping design or shouting from the touchline? [Keynote]. 13th International Conference of the Learning Sciences (ICLS ’18), 23–27 June 2018, London, UK. Retrieved from https://opus.lib.uts.edu.au/bitstream/10453/133232/1/Transitioning%20Educations%20Knowledge%20Infrastructure%20Shaping.pdf

Buckingham Shum, S., & Luckin, R. (2019). Learning analytics and AI: Politics, pedagogy, and practices. British Journal of Educational Technology, 50(6), 2785–2793. https://doi.org/10.1111/bjet.12880

Buckingham Shum, S., Ferguson, R., & Martinez-Maldonado, R. (2019). Human-centred learning analytics. Journal of Learning Analytics, 6(2), 1–9. https://doi.org/10.18608/jla.2019.62.1

Cerratto, T., & Waern, Y. (2000). Chatting to learn and learning to chat: Experienced teachers learning in a collaborative virtual environment. M/C Journal, 3(4). https://doi.org/10.5204/mcj.1866

Cerratto Pargman, T. (2006). Pour une conception des technologies centrée sur l’activité du sujet. Le cas de l’écriture de groupe avec collecticiel [Towards the design of technologies centered on the activity of the subject. The case of collaborative writing with groupware]. In P. Rabardel & P. Pastré (Eds.), Dans modèles du sujet pour la conception: Dialectiques activités développement (pp. 157–188). Paris, France: Octarès.

Cerratto Pargman, T. (2019). Unpacking emergent teaching practices with digital technology. In T. Cerratto Pargman & I. Jahnke (Eds.), Emergent practices and material conditions in learning and teaching with technologies (pp. 33–51). Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-030-10764-2_3

Cerratto Pargman, T. (2020) Practice as a concept in educational technology. In M. Peters & R. Heraud (Eds.), Encyclopedia of educational innovation. Singapore: Springer https://doi.org/10.1007/978-981-13-2262-4_146-2

Cerratto Pargman, T. C., & Jahnke, I. (Eds.). (2019). Emergent practices and material conditions in learning and teaching with technologies. Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-030-10764-2

Chen, B., & Zhu, H. (2019). Towards value-sensitive learning analytics design. Proceedings of the 9th International Conference on Learning Analytics and Knowledge (LAK ’19), 4–8 March 2019, Tempe, AZ, USA (pp. 343–352). New York, NY: ACM. https://doi.org/10.1145/3303772.3303798

Cormack, A. (2016). A data protection framework for learning analytics. Journal of Learning Analytics, 3(1), 91–106. https://doi.org/10.18608/jla.2016.31.6

Daniel, B. K. (2019). Big data and data science: A critical review of issues for educational research. British Journal of Educational Technology, 50(1), 101–113. https://doi.org/10.1111/bjet.12595

Denscombe, M. (2014). The good research guide: For small-scale social research projects, 5th edition. Maidenhead, Berkshire, UK: Open University Press.

Drachsler, H., & Greller, W. (2012). The pulse of learning analytics understandings and expectations from the stakeholders. In S. Buckingham Shum, D. Gašević, & R. Ferguson (Eds.), Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK ʼ12), 29 April–2 May 2012, Vancouver, BC, Canada (pp. 120–129). New York, NY: ACM. https://doi.org/10.1145/2330601.2330634

Drachsler, H., & Greller, W. (2016). Privacy and analytics — It’s a DELICATE issue. A checklist for trusted learning analytics. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼ16), 25–29 April 2016, Edinburgh, UK (pp. 89–98). New York, NY: ACM. https://doi.org/10.1145/2883851.2883893

Drachsler, H., Hoel, T., Scheffel, M., Kismihók, G., Berg, A., Ferguson, R., Kismihók, G., & Manderveld, J. (2015). Ethical and privacy issues in the application of learning analytics. Proceedings of the 5th International Conference on Learning Analytics and Knowledge (LAK ʼ15), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 390–391). New York, NY: ACM. https://doi.org/10.1145/2723576.2723642

D’Ignazio, C., & Klein, L. F. (2020). Data feminism. London, UK: MIT Press.

Engelfriet, A., Manderveld, J., & Jeunink, E. (2015). Learning analytics under the (Dutch) Personal Data Protection Act. Policy paper. SURFnet. Retrieved from https://www.surf.nl/binaries/content/assets/surf/nl/kennisbank/2015/surf_learning-analytics-onder-de-wet-wpb.pdf

Ferguson, R. (2019). Ethical challenges for learning analytics. Journal of Learning Analytics, 6(3), 25–30. https://doi.org/10.18608/jla.2019.63.5

Ferguson, R., & Clow, D. (2017). Where is the evidence? A call to action for learning analytics. Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 56–65). New York, NY: ACM. https://doi.org/10.1145/3027385.3027396

Ferguson, R., Hoel, T., Scheffel, M., & Drachsler, H. (2016). Guest editorial: Ethics and privacy in learning analytics. Journal of Learning Analytics, 3(1), 5–15. https://doi.org/10.18608/jla.2016.31.2

Glover, J. (2012). Humanity: A moral history of the twentieth century. New Haven, CT: Yale University Press.

Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26(2), 91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x

Heath, J. (2014). Contemporary privacy theory contributions to learning analytics. Journal of Learning Analytics, 1(1), 140–149. https://doi.org/10.18608/jla.2014.11.8

Howell, J. A., Roberts, L. D., Seaman, K., & Gibson, D. C. (2018). Are we on our way to becoming a “Helicopter University”? Academics’ views on learning analytics. Technology, Knowledge and Learning, 23(1). https://doi.org/10.1007/s10758-017-9329-9

Ifenthaler, D., & Schumacher, C. (2016). Student perceptions of privacy principles for learning analytics. Educational Technology Research and Development, 64(5, SI), 923–938. https://doi.org/10.1007/s11423-016-9477-y

Johnson, J. A. (2018). Toward information justice: Technology, politics, and policy for data in higher education administration (Vol. 33). Cham, Switzerland: Springer.

Jones, K. M. (2019a). Advising the whole student: eAdvising analytics and the contextual suppression of advisor values. Education and Information Technologies, 24(1), 437–458. https://doi.org/10.1007/s10639-018-9781-8

Jones, K. M. (2019b). Just because you can doesn’t mean you should: Practitioner perceptions of learning analytics ethics. Portal, 19(3), 407–428. https://doi.org/10.1353/pla.2019.0025

Jones, K. M., & VanScoy, A. (2019). The syllabus as a student privacy document in an age of learning analytics. Journal of Documentation, 75(6), 1333–1355. https://doi.org/10.1108/JD-12-2018-0202

Khalil, M., & Ebner, M. (2016). De-identification in learning analytics. Journal of Learning Analytics, 3(1), 129–138. https://doi.org/10.18608/jla.2016.31.8

Kim, D., Yoon, M., Jo, I. H., & Branch, R. M. (2018). Learning analytics to support self-regulated learning in asynchronous online courses: A case study at a women’s university in South Korea. Computers & Education, 127, 233–251. https://doi.org/10.1016/j.compedu.2018.08.023

Kitto, K., & Knight, S. (2019). Practical ethics for building learning analytics. British Journal of Educational Technology, 50(6), 2855–2870. https://doi.org/10.1111/bjet.12868

Klein, C., Lester, J., Rangwala, H., & Johri, A. (2019). Technological barriers and incentives to learning analytics adoption in higher education: Insights from users. Journal of Computing in Higher Education, 31(3), 604–625. https://doi.org/10.1007/s12528-019-09210-5

Knight, S., Buckingham Shum, S., & Littleton, K. (2014). Epistemology, assessment, pedagogy: Where learning meets analytics in the middle space. Journal of Learning Analytics, 1(2), 23–47. https://doi.org/10.18608/jla.2014.12.3

Knight, S., Buckingham Shum, S., Ryan, P., Sándor, Á., & Wang, X. (2018). Designing academic writing analytics for civil law student self-assessment. International Journal of Artificial Intelligence in Education, 28, 1–28. https://doi.org/10.1007/s40593-016-0121-0

Lawson, C., Beer, C., Rossi, D., Moore, T., & Fleming, J. (2016). Identification of ‘at risk’ students using learning analytics: The ethical dilemmas of intervention strategies in a higher education institution. Educational Technology Research and Development, 64, 957–968. https://doi.org/10.1007/s11423-016-9459-0

Mahroeian, H., Daniel, B., & Butson, R. (2017). The perceptions of the meaning and value of analytics in New Zealand higher education institutions. International Journal of Educational Technology in Higher Education, 14, 35. https://doi.org/10.1186/s41239-017-0073-y

Nissenbaum, H. (2004). Privacy as contextual integrity. Washington Law Review, 79, 119. Retrieved from https://digitalcommons.law.uw.edu/wlr/vol79/iss1/10

Nissenbaum, H. (2011). A contextual approach to privacy online. Daedalus, 140(4), 32–48. https://doi.org/10.1162/DAED_a_00113

Oblinger, D. G. (2012). Let’s talk... Analytics. Educause Review, 47(4), 10–13. Retrieved from https://er.educause.edu/articles/2012/7/lets-talk--analytics

Pardo, A., & Siemens, G. (2014). Ethical and privacy principles for learning analytics. British Journal of Educational Technology, 45(3), 438–450. https://doi.org/10.1111/bjet.12152

Petticrew, M., & Roberts, H. (2008). Systematic reviews in the social sciences: A practical guide. Victoria, Australia: Blackwell. https://doi.org/10.1080/14733140600986250

Prinsloo, P., & Slade, S. (2014). Educational triage in open distance learning: Walking a moral tightrope. International Review of Research in Open and Distributed Learning, 15(4), 306–331. Retrieved from https://files.eric.ed.gov/fulltext/EJ1039846.pdf

Prinsloo, P., & Slade, S. (2015). Student privacy self-management: Implications for learning analytics. Proceedings of the 5th International Conference on Learning Analytics and Knowledge (LAK ʼ15), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 83–92). New York, NY: ACM. https://doi.org/10.1145/2723576.2723585

Prinsloo, P., & Slade, S. (2016). Student vulnerability, agency, and learning analytics: An exploration. Journal of Learning Analytics, 3(1), 159–182. https://doi.org/10.18608/jla.2016.31.10

Prinsloo, P., & Slade, S. (2017a). Big data, higher education and learning analytics: Beyond justice, towards an ethics of care. In Daniel B. Kei (Ed.), Big data and learning analytics in higher education (pp.109–124). Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-319-06520-5_8

Prinsloo, P., & Slade, S. (2017b). An elephant in the learning analytics room: The obligation to act. Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 46–55). New York, NY: ACM. https://doi.org/10.1145/3027385.3027406

Puig de La Bellacasa, M. (2011). Matters of care in technoscience: Assembling neglected things. Social Studies of Science, 41(1), 85–106. https://doi.org/10.1177/0306312710380301

Reidenberg, J. R., & Schaub, F. (2018). Achieving big data privacy in education. Theory and Research in Education, 16(3), 263–279. https://doi.org/10.1177/1477878518805308

Roberts, L. D., Howell, J. A., Seaman, K., & Gibson, D. C. (2016). Student attitudes toward learning analytics in higher education: The Fitbit version of the learning world. Frontiers in Psychology, 7, 1959. https://doi.org/10.3389/fpsyg.2016.01959

Sclater, N. (2016). Developing a code of practice for learning analytics. Journal of Learning Analytics, 3(1), 16–42. https://doi.org/10.18608/jla.2016.31.3

Sclater, N., & Bailey, P. (2015). Code of practice for learning analytics. Retrieved from https://www.jisc.ac.uk/guides/code-of-practice-for-learning-analytics

Scott, J., & Nichols, T. P. (2017). Learning analytics as assemblage: Criticality and contingency in online education. Research in Education, 98(1), 83–105. https://doi.org/10.1177/0034523717723391

Selwyn, N. (2019). What’s the problem with learning analytics? Journal of Learning Analytics, 6(3), 11–19. https://doi.org/10.18608/jla.2019.63.3

Siemens, G., & Long, P. (2011). Penetrating the fog: Analytics in learning and education. EDUCAUSE Review, 46(5), 30. Retrieved from https://er.educause.edu/articles/2011/9/penetrating-the-fog-analytics-in-learning-and-education

Slade, S., & Prinsloo, P. (2013). Learning analytics: Ethical issues and dilemmas. American Behavioral Scientist, 57(10), 1510–1529. https://doi.org/10.1177/0002764213479366

Slade, S., & Prinsloo, P. (2014). Student perspectives on the use of their data: Between intrusion, surveillance and care. Proceedings of the European Distance and E-Learning Network 2014 Research Workshop (EDEN 2014), 27–28 October 2014, Oxford, UK (pp. 291–300).

Slade, S., Prinsloo, P., & Khalil, M. (2019). Learning analytics at the intersections of student trust, disclosure and benefit. Proceedings of the 9th International Conference on Learning Analytics and Knowledge (LAK ’19), 4–8 March 2019, Tempe, AZ, USA (pp. 235–244). New York, NY: ACM. https://doi.org/10.1145/3303772.3303796

Sun, K., Mhaidli, A. H., Watel, S., Brooks, C. A., & Schaub, F. (2019). It’s my data! Tensions among stakeholders of a learning analytics dashboard. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19), 4–9 May 2019, Glasgow, Scotland, UK (Paper 594, pp. 1–14). New York, NY: ACM. https://doi.org/10.1145/3290605.3300824

Swenson, J. (2014). Establishing an ethical literacy for learning analytics. Proceedings of the 4th International Conference on Learning Analytics and Knowledge (LAK ʼ14), 24–28 March 2014, Indianapolis, IN, USA (pp. 246–250). New York, NY: ACM. https://doi.org/10.1145/2567574.2567613

Tsai, Y. S., Poquet, O., Gašević, D., Dawson, S., & Pardo, A. (2019). Complexity leadership in learning analytics: Drivers, challenges and opportunities. British Journal of Educational Technology, 50(6), 2839–2854. https://doi.org/10.1111/bjet.12846

Verbeek, P.-P. (2005). What things do: Philosophical reflections on technology, agency, and design. University Park, PA, USA: Penn State Press.

Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in Human Behavior, 89, 98–110. https://doi.org/10.1016/j.chb.2018.07.027

Wessel, M., Helgesson, G., Olsson, D., Juth, N., Alexanderson, K., & Lynöe, N. (2013). When do patients feel wronged? Empirical study of sick-listed patients’ experiences with healthcare encounters. The European Journal of Public Health, 23(2), 230–235. https://doi.org/10.1093/eurpub/cks030

West, D., Huijser, H., & Heath, D. (2016). Putting an ethical lens on learning analytics. Educational Technology Research and Development, 64, 903–922. https://doi.org/10.1007/s11423-016-9464-3

West, D., Tasir, Z., Luzeckyj, A., Kew, S. N., Toohey, D., Abdullah, Z., Searle, B., Nurul, F. H., & Price, R. (2018). Learning analytics experience among academics in Australia and Malaysia: A comparison. Australian Journal of Educational Technology, 34(3), 122–139. https://doi.org/10.14742/ajet.3836

Whitelock‐Wainwright, A., Gašević, D., Tejeiro, R., Tsai, Y., & Bennett, K. (2019). The student expectations of learning analytics questionnaire. Journal of Computer Assisted Learning, 35(5), 633–666. https://doi.org/10.1111/jcal.12366

Willis, J. E. (2014, August 25). Learning analytics and ethics: A framework beyond utilitarianism. Educause Review. Retrieved from https://er.educause.edu/articles/2014/8/learning-analytics-and-ethics-a-framework-beyond-utilitarianism

Willis, J. E., Pistilli, M. D., & Campbell, J. P. (2013). Ethics, big data, and analytics: A model for application. Educause Review Online. Retrieved from https://docs.lib.purdue.edu/idcpubs/1/

Willis, J. E., Slade, S., & Prinsloo, P. (2016). Ethical oversight of student data in learning analytics: A typology derived from a cross-continental, cross-institutional perspective. Educational Technology Research and Development, 64, 881–901. https://doi.org/10.1007/s11423-016-9463-4

Yin, R. K. (2017). Case study research and applications: Design and methods. Thousand Oaks, CA, USA: Sage Publications.

Mapping the Ethics of Learning Analytics in Higher Education

A Systematic Literature Review of Empirical Research

Authors

DOI:

Keywords:

Abstract

References

Downloads

Published

How to Cite

Issue

Section

License

Most read articles by the same author(s)