What Does Methodology Mean for Learning Analytics?
DOI:
https://doi.org/10.18608/jla.2018.52.1Keywords:
Methodology, epistemology, evaluation, sensitivity analysis, bridge disciplineAbstract
The diversity of substantive expertise and analytical techniques encompassed by learning analytics can be a challenge for understanding and interrogating methodological choices. But active pursuit of methodological clarity is essential to productive multi-vocality. This was the focus of the LAK ’17 Methodology in Learning Analytics workshop from which the idea for this special section arose. Discussions were framed using a building metaphor of developing structurally sound bridges between disciplinary methods. Strong arguments are buttressed by supporting evidence and warrants but also engage with potential alternative explanations. Papers in this special section further the discussion and span a range of relevant approaches including sensitivity analysis, model evaluation and model fit, causal inference, and visualisation as a methodology. All advance the case of structural integrity in this bridge discipline and highlight the importance of adequate descriptions of methodological considerations to aid our interrogation of methodology and epistemology.
References
Bergner, Y. (2017) Measurement and its uses in learning analytics. In Lang, C., Siemens, G., Wise, A., & Gašević, D. (Eds.), Handbook of Learning Analytics (pp. 34-48). SOLAR, Society for Learning Analytics Research.
Bergner, Y., Kerr, D., and Pritchard, D. E. (2015). Methodological Challenges in the Analysis of MOOC Data for Exploring the Relationship between Discussion Forum Views and Learning Outcomes. Proceedings of 8th International Conference on Educational Data Mining, 26-29 June 2015, Madrid, Spain (pp. 234-241).
Bergner, Y., Lang, C. & Gray, G. (2017a) Workshop on Methodology in Learning Analytics. In Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ‘17), 13–17 March 2017, Vancouver, BC, Canada (pp. 500-501). New York: ACM. https://doi.org/10.1145/3027385.3029427
Bergner, Y., Lang, C., & Gray, G. (2017b). A Focus on Methodology in Learning Analytics: Building a Structurally Sound Bridge Discipline. In CEUR Workshop Proceedings (Vol. 1915).
Box, G. E. (1979). Robustness in the strategy of scientific model building. In Launer & Wilkinson (Eds.), Robustness in statistics (pp. 201-236). Academic Press.
Brooks, C., Chavez, O., Tritz, J., & Teasley, S. (2015). Reducing selection bias in quasi-experimental educational studies. Proceedings of the Fifth International Conference on Learning Analytics and Knowledge (LAK’15), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 295-299). New York: ACM. https://doi.org/10.1145/2723576.2723614
Clow, D. (2013) An overview of learning analytics. Teaching in Higher Education, 18(6), 683–695.
Gray, G., McGuinness, C., Owende, P., McCarthy, A. (2014). A review of psychometric data analysis and applications in modelling of academic achievement in tertiary education. Journal of Learning Analytics, 1(1), 75-106.
Hand, D. J. (1998). Data mining: Statistics and more? The American Statistician, 52(2), 112-118.
Imai, K., King, G., & Stuart, E. (2008). Misunderstandings between experimentalists and observationalists about causal inference. Journal of the Royal Statistical Society: Series A (Statistics in Society), 171(2), 481-502. https://doi.org/10.1111/j.1467-985X.2007.00527.x
Kennedy, P. E. (2002). Sinning in the basement: what are the rules? The ten commandments of applied econometrics. Journal of Economic Surveys, 16(4), 569–589. https://doi.org/10.1111/1467-6419.00179
Knight, S., Buckingham Shum, S., & Littleton, K. (2013, April). Epistemology, pedagogy, assessment and learning analytics. In Proceedings of the Third International Conference on Learning Analytics and Knowledge, 8-12 April 2013, Leuven, Belgium (pp. 75-84). New York: ACM. https://doi.org/10.1145/2460296.2460312
Kovanović, V., Gašević, D., Dawson, S., Joksimović, S., Baker, R.S.J.D., and Hatala, M. (2015). Penetrating the black box of time-on-task estimation. Proceedings of the Fifth International Conference on Learning Analytics and Knowledge (LAK’15), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 184–193). New York: ACM. https://doi.org/10.1145/2723576.2723623
Lakoff, G., & Johnson, M. (1980) Metaphors we live by. Chicago: University of Chicago Press.
MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. D. (2002). On the practice of dichotomization of quantitative variables. Psychological Methods, 7(1), 19–40. https://doi.org/10.1037/1082-989X.7.1.19
Martin, B., Mitrovic, A., Koedinger, K. R., & Mathan, S. (2011). Evaluating and improving adaptive educational systems with learning curves. User Modeling and User-Adapted Interaction, 21(3), 249-283.
Pelánek, R. (2015). Metrics for evaluation of student models. Journal of Educational Data Mining, 7(2), 1-19.
Pelánek, R., Rihák, J., and Papoušek, J. (2016). Impact of data collection on interpretation and evaluation of student models. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼ16), 25–29 April 2016, Edinburgh, UK (pp. 40–47). New York: ACM. https://doi.org/10.1145/2883851.2883868
Richardson, M., Abraham, C., & Bond, R. (2012). Psychological correlates of university students' academic performance: A systematic review and meta-analysis. Psychological bulletin, 138(2), 353-387.
Ritter, S., Anderson, J. R., Koedinger, K. R., & Corbett, A. (2007). Cognitive tutor: applied research in mathematics education. Psychonomic Bulletin & Review, 14(2), 249–255. https://doi.org/10.3758/BF03194060
Siemens, G., & Long, P. (2011). Penetrating the fog: Analytics in learning and education. EDUCAUSE review, 46(5), 30.
Suthers, D., & Verbert, K. (2013). Learning analytics as a middle space. In Proceedings of the Third International Conference on Learning Analytics and Knowledge, 8-12 April 2013, Leuven, Belgium (pp. 2.5). New York: ACM. https://doi.org/10.1145/2460296.2460298
Tempelaar, D. T., Rienties, B., & Nguyen, Q. (2017). Towards actionable learning analytics using dispositions. IEEE Transactions on Learning Technologies, 10(1), 6-16.
Toulmin, S. E. (2003). The Uses of Argument (Updated edition 2003). Cambridge University Press.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2018 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
