Learning Analytics in Physics Education
Equity-Focused Decision-Making Lacks Guidance!
DOI:
https://doi.org/10.18608/jla.2023.7793Keywords:
equity, principles, ethics, bias, critical consciousness, learning progression, responsible learning analytics, research paperAbstract
Learning Analytics are an academic field with promising usage scenarios for many educational domains. At the same time, learning analytics come with threats such as the amplification of historically grown inequalities. A range of general guidelines for more equity-focused learning analytics have been proposed but fail to provide sufficiently clear guidance for practitioners. With this paper, we attempt to address this theory–practice gap through domain-specific (physics education) refinement of the general guidelines We propose a process as a starting point for this domain-specific refinement that can be applied to other domains as well. Our point of departure is a domain-specific analysis of historically grown inequalities in order to identify the most relevant diversity categories and evaluation criteria. Through two focal points for normative decision-making, namely equity and bias, we analyze two edge cases and highlight where domain-specific refinement of general guidance is necessary. Our synthesis reveals a necessity to work towards domain-specific standards and regulations for bias analyses and to develop counter-measures against (intersectional) discrimination. Ultimately, this should lead to a stronger equity-focused practice in future.
References
Aikins, M. A., Bremberger, T., Aikins, J. K., Gyamerah, D., & Yıldırım-Caliman, D. (2021). Afrozensus 2020: Perspektiven, Anti-Schwarze Rassismuserfahrungen und Engagement Schwarzer, afrikanischer und afrodiasporischer Menschen in Deutschland. https://afrozensus.de
Artificial intelligence, platform work and gender equality. (2022). European Institute for Gender Equality. https://doi.org/10.2839/372863
Avraamidou, L. (2019). “I am a young immigrant woman doing physics and on top of that I am Muslim”: Identities, intersections, and negotiations. Journal of Research in Science Teaching, 57, 311–341. https://doi.org/10.1002/tea.21593
Baggett, H. C. (2020). Relevance, Representation, and Responsibility: Exploring World Language Teachers’ Critical Consciousness and Pedagogies. L2 Journal, 12(2), 34–54. https://doi.org/10.5070/L212246037
Baker, R., & Hawn, A. (2021). Algorithmic Bias in Education. https://doi.org/10.1007/s40593-021-00285-9
Bergner, Y. (2017). Chapter 3: Measurement and its Uses in Learning Analytics. In Handbook of Learning Analytics (1st ed., pp. 35–48). SoLAR. https://doi.org/10.18608/hla17.003
Broughan, C., & Prinsloo, P. (2020). (Re)centring students in learning analytics: In conversation with Paulo Freire. Assessment & Evaluation in Higher Education, 45(4), 617–628. https://doi.org/10.1080/02602938.2019.1679716
Carlone, H. B., & Johnson, A. (2007). Understanding the Science Experiences of Succesful Women of Color: Science Identity as an Analytic Lens. Journal of Research in Science Teaching, 44(8), 1187–1218. https://doi.org/10.1002/tea.20237
Cerratto Pargman, T., & McGrath, C. (2021). Mapping the Ethics of Learning Analytics in Higher Education: A Systematic Literature Review of Empirical Research. Journal of Learning Analytics, 8(2), 123–139. https://doi.org/10.18608/jla.2021.1
Cerratto Pargman, T., McGrath, C., Viberg, O., Kitto, K., Knight, S., & Ferguson, R. (2021). Responsible Learning Analytics: Creating just, ethical, and caring LA systems. Companion Proceedings. LAK21. https://www.solaresearch.org/wp-content/uploads/2021/04/LAK21_CompanionProceedings.pdf
Cheuk, T. (2021). Can AI be racist? Color‐evasiveness in the application of machine learning to science assessments. Science Education, 1–12. https://doi.org/10.1002/sce.21671
Costanza-Chock, S. (2020). Design justice: Community-led practices to build the worlds we need. The MIT Press.
D’Ignazio, C., & Klein, L. (2020). Introduction: Why Data Science Needs Feminism. In Data Feminism. https://data-feminism.mitpress.mit.edu/pub/frfa9szd/release/6
Dou, R., Hazari, Z., Dabney, K., Sonnert, G., & Sadler, P. (2019). Early informal STEM experiences and STEMidentity: The importance of talking science. Science Education, 103, 623–637. https://doi.org/10.1002/sce.21499
Drachsler, H., & Greller, W. (2016). Privacy and Analytics – it’s a DELICATE Issue. LAK 16, Edinburgh. http://dx.doi.org/10.1145/2883851.2883893
Düchs, G., & Ingold, G.-L. (2018). Frauenanteil bleibt stabil. Physik Journal, 17(8/9), 32–37.
Duncan, R. G., & Rivet, A. E. (2018). Learning progressions. In F. Fischer, C. E. Hmelo-Silver, S. R. Goldman, & P. Reimann (Eds.), International Handbook of the Learning Sciences (pp. 422–432). Routledge.
El-Mafaalani, A. (2021). Mythos Bildung (2nd ed.). Kiepenheuer & Witsch (KiWi).
Erden, D. (2020). KI und Beschäftigung: Das Ende menschlicher Vorurteile oder der Beginn von Diskriminierung 2.0? In Wenn KI, dann feministisch (pp. 77–90). netzforma* eV. https://netzforma.org/publikation-wenn-ki-dann-feministisch-impulse-aus-wissenschaft-und-aktivismus
Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., Schafer, B., Valcke, P., & Vayena, E. (2018). AI4People—An Ethical Framework for a Good AI Society: Opportunities, Risks, Principles, and Recommendations. Minds & Machines, 28(4), 689–707. https://doi.org/10.1007/s11023-018-9482-5
Francis, P., Broughan, C., Foster, C., & Wilson, C. (2020). Thinking critically about learning analytics, student outcomes, and equity of attainment. Assessment & Evaluation in Higher Education, 45(6), 811–821. https://doi.org/10.1080/02602938.2019.1691975
Freire, P. (1970). Pedagogy of the Oppressed. Penguin Random House UK.
Gardner, J., Brooks, C., & Baker, R. (2019). Evaluating the Fairness of Predictive Student Models Through Slicing Analysis. LAK19: Proceedings of the 9th International Conference on Learning Analytics & Knowledge, 225–234. https://doi.org/10.1145/3303772.3303791
Godwin, A. (2016). The Development of a Measure of Engineering Identity. 2016 ASEE Annual Conference & Exposition, New Orleans, Louisiana. https://doi.org/10.18260/p.26122
Gutachten der Datenethikkommission der Bundesregierung (pp. 1–32). (2019). [Kurzfassung]. DEK. https://www.bmi.bund.de/SharedDocs/downloads/DE/publikationen/themen/it-digitalpolitik/gutachten-datenethikkommission-kurzfassung.pdf?__blob=publicationFile&v=4
Khalil, M., Prinsloo, P., & Slade, S. (2022). A Comparison of Learning Analytics Frameworks: A Systematic Review. LAK22: LAK22: 12th International Learning Analytics and Knowledge Conference, 152–163. https://doi.org/10.1145/3506860.3506878
Kitto, K., & Knight, S. (2019). Practical ethics for building learning analytics. British Journal of Educational Technolog, 50(6), 2855–2870. https://doi.org/10.1111/bjet.12868
Kracke, N., Buck, D., & Middendorff, E. (2018). Beteiligung an Hochschulbildung, Chancen(un)gleichheit in Deutschland. DZHW Brief, 3. https://doi.org/10.34878/2018.03.dzhw_brief
Lohaus, M., Perrot, M., & von Luxburg, U. (2020). Too Relaxed to Be Fair. Proceedings of Machine Learning Research, 119, 6360--6369. https://proceedings.mlr.press/v119/lohaus20a.html
Mitchell, S., Potash, E., D´Amour, A., & Lum, K. (2021). Algorithmic Fairness: Choices, Assumptions, and Definitions. Annual Review of Statistics and Its Application, 8, 141–163. https://doi.org/10.1146/annurev-statistics-042720-125902
Mujtaba, T., & Reiss, M. J. (2013). Inequality in Experiences of Physics Education: Secondary School Girls’ and Boys’ Perceptions of their Physics Education and Intentions to Continue with Physics After the Age of 16. International Journal of Science Education, 35(11), 1824–1845. https://doi.org/10.1080/09500693.2012.762699
Neue deutsche Medienmacher (NdM)—Glossar. (2021). https://glossar.neuemedienmacher.de/
Odoi, N. (2021). Die Farbe der Gerechtigkeit ist weiß | bpb. bpb.de. https://www.bpb.de/gesellschaft/migration/afrikanische-diaspora/59470/rassismus-im-strafrechtssystem
OECD. (2016). Excellence and equity in education (Volume I; PISA 2015 Results). OECD.
OECD. (2018). Where All Students Can Succeed (Volume II; PISA 2018 Results). OECD.
Ogette, T. (2019). Exit racism (5th ed.). unrast-Verlag.
Phillips, P. J., Hahn, C. A., Fontana, P. C., Broniatowski, D. A., & Przybocki, M. A. (2020). Four Principles of Explainable Artificial Intelligence. National Institute of Standards and Technology. https://doi.org/10.6028/NIST.IR.8312-draft
Prinsloo, P., & Slade, S. (2018). Mapping responsible learning analytics: A critical proposal. In Responsible Analytics & Data Mining in Education: Global Perspectives on Quality, Support, and Decision-Making. Routledge.
Rainey, K., Dancy, M., Mickelson, R., Stearns, E., & Moller, S. (2018). Race and gender differences in how senseof belonging influences decisions to majorin STEM. International Journal of STEM Education, 5. https://doi.org/10.1186/s40594-018-0115-6
Rosa, K., & Moore Mensah, F. (2016). Educational pathways of Black women physicists: Stories of experiencing and overcoming obstacles in life. Physical Review Physics Education Research, 12(2), Article 2. https://doi.org/10.1103/PhysRevPhysEducRes.12.020113
Sclater, N. (2014). Code of practice for learning analytics (pp. 1–64) [Literature review]. Jisc. https://repository.jisc.ac.uk/5661/1/Learning_Analytics_A-_Literature_Review.pdf
Slack, D., Hilgard, H., Jia, E., Singh, S., & Lakkaraju, H. (2020). Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods. Proceedings of the 2020 AAAI/ACM Conference on AI, Ethics, and Society (AIES ’20), 180–186. https://doi.org/10.1145/3375627.3375830
Slade, S. (2016). The Open University Ethical use of Student Data for Learning Analytics Policy. The Open University. https://doi.org/10.13140/RG.2.1.1317.4164
Smith, T., Avraamidou, L., & Adams, J. D. (2022). Culturally relevant/responsive and sustaining pedagogies in science education: Theoretical perspectives and curriculum implications. https://doi.org/10.1007/s11422-021-10082-4
SoLAR. (2022). What is Learning Analytics? Society for Learning Analytics Research (SoLAR). https://www.solaresearch.org/about/what-is-learning-analytics/
Steegh, A. M., Höffler, T. N., Keller, M. M., & Parchmann, I. (2019). Gender differences in mathematics and science competitions: A systematic review. Journal of Research in Science Teaching, 56(10), Article 10. https://doi.org/10.1002/tea.21580
Suresh, H., & Guttag, J. (2021). A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle. EAAMO ’21: Equity and Access in Algorithms, Mechanisms, and Optimization, 1–9. https://doi.org/10.1145/3465416.3483305
Traag, V. A., & Waltman, L. (2022). Causal foundations of bias, disparity and fairness. ArXiv. https://doi.org/10.48550/arXiv.2207.13665
Wise, A. F., Sarmiento, J. P., & Boothe Jr., M. (2021). Subversive Learning Analytics. 639–645. https://doi.org/10.1145/3448139.3448210
Yeung, K. (2019). Responsibility and AI (DGI(2019)05; Issue DGI(2019)05). Council of Europe. https://rm.coe.int/responsability-and-ai-en/168097d9c5
Zhai, X., Haudek, K. C., Shi, L., Nehm, R. H., & Urban-Lurain, M. (2019). From substitution to redefinition: A framework of machine learning-based science assessment. Journal of Research in Science Teaching, 57, 1430–1459. https://doi.org/10.1002/tea.21658
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
