Towards data-informed teaching practice:
A model for integrating analytics with teacher inquiry
DOI:
https://doi.org/10.18608/jla.2022.7505Keywords:
data-informed teacher inquiry, teaching analytics, learning analytics, teacher inquiry model, thematic analysis, research paperAbstract
Data-informed decision making in teachers’ practice, now recommended by different teacher inquiry models and policy documents, implies deep practice change for many teachers. However, not much is known how teachers perceive the different steps that analytics-informed teacher inquiry into their own practice entails. This paper presents the results of a study into developing an Analytics Model for Teacher Inquiry (AMTI), which was then used to understand how teachers (N=10) construe the steps in the model and to explore the possible constraints as well as incentives for Teaching and Learning Analytics (TLA)-informed teacher practices. In the final iteration experts (N=7) and teacher-researchers (N=2) tested and evaluated the developed model. Their feedback was used to improve the model and provide example cases with insights into possible scenarios for TLA-informed analyses of teaching.
References
ALLEA, A. (2017). The European Code of Conduct for Research Integrity, revised edition. Berlin2017. Retrieved from: http://www. allea. org. https://allea.org/code-of-conduct/
Alumäe, T., & Tilk, O. (2019). Advanced Rich Transcription System for Estonian Speech. arXiv preprint arXiv:1901.03601. https://doi.org/10.3233/978-1-61499-912-6-1
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa
Brown, C., Schildkamp, K., & Hubers, M. D. (2017). Combining the best of two worlds: A conceptual proposal for evidence-informed school improvement. Educational research, 59(2), 154-172. https://doi.org/10.1080/00131881.2017.1304327
Burks, B. A., Beziat, T. L., Danley, S., Davis, K., Lowery, H., & Lucas, J. (2015). Adapting to change: Teacher perceptions of implementing the Common Core State Standards. Education, 136(2), 253-258. Retrieved from: https://www.ingentaconnect.com/content/prin/ed/2015/00000136/00000002/art00013
Clow, D. (2012, April). The learning analytics cycle: closing the loop effectively. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 134-138). https://doi.org/10.1145/2330601.2330636
Corrin, L., De Barba, P. G., Lockyear, L., Gašević, D., Williams, D., Dawson, S., Mulder, R., Copeland, S., & Bakharia, A. (2016). Completing the Loop: Returning Meaningful Learning Analytic Data to Teachers. Australian Government Office for Learning and Teaching. Retrieved from: https://ltr.edu.au/resources/ID13_3068_Kennedy_Report_2016.pdf
Creswell , J.W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (4th ed.). Boston: Pearson Education.
Dawson, S., Joksimovic, S., Poquet, O., & Siemens, G. (2019, March). Increasing the impact of learning analytics. In Proceedings of the 9th International Conference on Learning Analytics & Knowledge (pp. 446-455). https://doi.org/10.1145/3303772.3303784
Delbecq, A. L., Van de Ven, A. H., & Gustafson, D. H. (1975). Group techniques for program planning: A guide to nominal group and Delphi processes: Scott. Foresman Glenview, IL.
Ebbeler, J., Poortman, C. L., Schildkamp, K., & Pieters, J. M. (2016). Effects of a data use intervention on educators’ use of knowledge and skills. Studies in educational evaluation, 48, 19-31. https://doi.org/10.1016/j.stueduc.2015.11.002
Edström, K. (2008). Doing course evaluation as if learning matters most. Higher education research & development, 27(2), 95-106. https://doi.org/10.1080/07294360701805234
Ferguson, R., Macfadyen, L., Clow, D., Tynan, B., Alexander, S. and Dawson, S. (2014). Setting Learning Analytics in Context: Overcoming the Barriers to Large‐Scale Adoption. Journal of Learning Analytics, 1, 3, 120-144. https://doi.org/10.18608/jla.2014.13.7
Fullan, M. (Ed.). (2014). Teacher development and educational change. Routledge. https://doi.org/10.4324/9781315870700
Fuller, K. A., Karunaratne, N. S., Naidu, S., Exintaris, B., Short, J. L., Wolcott, M. D., ... & White, P. J. (2018). Development of a self-report instrument for measuring in-class student engagement reveals that pretending to engage is a significant unrecognized problem. PloS one, 13(10), e0205828. https://doi.org/10.1371/journal.pone.0205828
Gaertner, H. (2014). Effects of student feedback as a method of self-evaluating the quality of teaching. Studies in Educational Evaluation, 42, 91-99. https://doi.org/10.1016/j.stueduc.2014.04.003
Gleason, P., Crissey, S., Chojnacki, G., Zukiewicz, M., Silva, T., Costelloe, S., & O'Reilly, F. (2019). Evaluation of Support for Using Student Data to Inform Teachers' Instruction. NCEE 2019-4008. National Center for Education Evaluation and Regional Assistance. Retrieved from: https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=NCEE20194008
Goertz, M.E., Nabors Oláh, L. & Riggan, M. (2009). From testing to teaching: The use of interim assessments in classroom instruction. CPRE Research Report #RR-65. Philadelphia, PA: Consortium for Policy Research in Education. https://doi.org/10.12698/CPRE.2009.RR65
Greller, W., & Drachsler, H. (2012). Translating learning into numbers: A generic framework for learning analytics. Journal of Educational Technology & Society, 15(3), 42-57. Retrieved from: http://www.jstor.org/stable/jeductechsoci.15.3.42
Hansen, C. J., & Wasson, B. (2016). Teacher Inquiry into Student Learning:-The TISL Heart Model and Method for use in Teachers’ Professional Development. Nordic Journal of Digital Literacy, 11(01), 24-49. https://doi.org/10.18261/issn.1891-943x-2016-01-02
Hea Teadustava. (2017). Estonian Code of Conduct for Research Integrity. Tartu. Retrieved from: https://www.eetika.ee/sites/default/files/www_ut/hea_teadustava_eng_trukis.pdf
Ingram, D., Seashore Louis, K., & Schroeder, R. (2004). Accountability policies and teacher decision making: Barriers to the use of data to improve practice. Teachers college record, 106(6), 1258-1287. https://doi.org/10.1111/j.1467-9620.2004.00379.x
Joksimović, S., Kovanović, V., & Dawson, S. (2019). The journey of learning analytics. HERDSA Review of Higher Education, 6, 27-63. Retrieved from: https://www.herdsa.org.au/herdsa-review-higher-education-vol-6/37-63
Lai, M. K., & Schildkamp, K. (2013). Data-based decision making: An overview. In Data-based decision making in education (pp. 9-21). Springer, Dordrecht. https://doi.org/10.1007/978-94-007-4816-3
Mandinach, E. B. (2012). A perfect time for data use: Using data-driven decision making to inform practice. Educational Psychologist, 47(2), 71-85. https://doi.org/10.1080/00461520.2012.667064
Marsh, J. A., Pane, J. F., & Hamilton, L. S. (2006). Making sense of data-driven decision making in education: Evidence from recent RAND research. Retrieved from: https://www.rand.org/pubs/occasional_papers/OP170.html
Maulana, R., Helms-Lorenz, M., & Van de Grift, W. (2017). Validating a model of effective teaching behaviour of pre-service teachers. Teachers and Teaching, 23(4), 471-493. Retrieved from: https://psycnet.apa.org/record/2017-09817-007
McKenney, S., & Reeves, T. C. (2012). Conducting educational design research. London: Routledge.
Meyers, G. L., Jacobsen, M., & Henderson, E. (2018). Design-Based Research: Introducing an innovative research methodology to infection prevention and control. Canadian Journal of Infection Control, 33(3). Retrieved from: https://ipac-canada.org/photos/custom/CJIC/CJIC_Fall2018_Meyers.pdf
Molenaar, I., & Knoop-van Campen, C. A. (2018). How teachers make dashboard information actionable. IEEE Transactions on Learning Technologies, 12(3), 347-355. https://doi.org/10.1109/TLT.2018.2851585
Mullen, R., Kydd, A., Fleming, A., & McMillan, L. (2021). A practical guide to the systematic application of nominal group technique. Nurse researcher, 29(1). https://doi.org/10.7748/nr.2021.e1777
Ndukwe, I. G., & Daniel, B. K. (2020). Teaching analytics, value and tools for teacher data literacy: A systematic and tripartite approach. International Journal of Educational Technology in Higher Education, 17(1), 1-31. https://doi.org/10.1186/s41239-020-00201-6
Poortman, C. L., & Schildkamp, K. (2016). Solving student achievement problems with a data use intervention for teachers. Teaching and teacher education, 60, 425-433. https://doi.org/10.1016/j.tate.2016.06.010
Priestley, M., Edwards, R., Priestley, A., & Miller, K. (2012). Teacher agency in curriculum making: Agents of change and spaces for manoeuvre. Curriculum inquiry, 42(2), 191-214. https://doi.org/10.1111/j.1467-873X.2012.00588.x
Rolfe, G., Freshwater, D., & Jasper, M. (2001). Critical Reflection for Nursing and the Helping Professions a User's Guide. Basingstoke, Palgrave Macmillan
Schildkamp, K., & Datnow, A. (2020). When Data Teams Struggle: Learning from Less Successful Data Use Efforts. Leadership and Policy in Schools, 1-20. https://doi.org/10.1080/15700763.2020.1734630
Schildkamp, K., & Kuiper, W. (2010). Data-informed curriculum reform: Which data, what purposes, and promoting and hindering factors. Teaching and teacher education, 26(3), 482-496. doi.org/ https://doi.org/10.1016/j.tate.2009.06.007
Schildkamp, K., & Poortman, C. (2015). Factors influencing the functioning of data teams. Teachers college record, 117(4). Retrieved from: https://research.utwente.nl/en/publications/factors-influencing-the-functioning-of-data-teams
Schildkamp, K., Poortman, C. L., & Handelzalts, A. (2016). Data teams for school improvement. School effectiveness and school improvement, 27(2), 228-254. https://doi.org/10.1080/09243453.2015.1056192
Schildkamp, K., Poortman, C. L., Ebbeler, J., & Pieters, J. M. (2019). How school leaders can build effective data teams: Five building blocks for a new wave of data-informed decision making. Journal of educational change, 20(3), 283-325. https://doi.org/10.1007/s10833-019-09345-3
Seidel, T., Stürmer, K., Blomberg, G., Kobarg, M., & Schwindt, K. (2011). Teacher learning from analysis of videotaped classroom situations: Does it make a difference whether teachers observe their own teaching or that of others?. Teaching and teacher education, 27(2), 259-267. https://doi.org/10.1016/j.tate.2010.08.009
Sergis, S., & Sampson, D. G. (2017). Teaching and learning analytics to support teacher inquiry: A systematic literature review. Learning analytics: Fundaments, applications, and trends, 25-63. https://doi.org/10.1007/978-3-319-52977-6_2
Timperley, H. (2010, February). Using evidence in the classroom for professional learning. In Étude présentée lors du Colloque ontarien sur la recherche en éducation. Retrieved from: https://cdn.auckland.ac.nz/assets/education/about/schools/tchldv/docs/Using%20Evidence%20in%20the%20Classroom%20for%20Professional%20Learning.pdf
Varga-Atkins, T., McIsaac, J., & Willis, I. (2017). Focus Group meets Nominal Group Technique: an effective combination for student evaluation?. Innovations in Education and Teaching International, 54(4), 289-300. https://doi.org/10.1080/14703297.2015.1058721
Viberg, O., & Gronlund, A. (2021). Desperately seeking the impact of learning analytics in education at scale: Marrying data analysis with teaching and learning. Retrieved from: arXiv:2105.06680v1
Wang, F., & Hannafin, M. J. (2005). Design-based research and technology-enhanced learning environments. Educational technology research and development, 53(4), 5-23. Retrieved from: https://www.jstor.org/stable/30221206
Wayman, J. C., & Jimerson, J. B. (2014). Teacher needs for data-related professional learning. Studies in Educational Evaluation, 42, 25-34. https://doi.org/10.1016/j.stueduc.2013.11.001
Wise, A. F., & Jung, Y. (2019). Teaching with analytics: Towards a situated model of instructional decision-making. Journal of Learning Analytics, 6(2), 53-69. https://doi.org/10.18608/jla.2019.62.4
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
