ADA for IBL: Lessons Learned in Aligning Learning Design and Analytics for Inquiry-Based Learning Orchestration
DOI:
https://doi.org/10.18608/jla.2021.7357Keywords:
learning design, learning analytics, inquiry-based learning, orchestration, technology-enhanced learningAbstract
Orchestrating technology-enhanced learning is a difficult task, especially in demanding pedagogical approaches like inquiry-based learning (IBL). To foster effective teacher adoption, both the complexity of designing IBL activities and the uncertainty about the student learning path during enactment need to be addressed. Previous research suggests that aligning learning design and learning analytics can be an effective way to provide such orchestration support. This paper reports on a design-based research (DBR) project to explore teachers’ orchestration needs in Go-Lab (a technological ecosystem for IBL used by thousands of primary and secondary school teachers around the world), and on how solutions that align learning design and analytics can fulfill such needs. The analysis of data from multiple events (including surveys, case studies, workshops with teachers, and platform usage analyses) led to a catalogue of IBL orchestration needs that can be tackled by aligning learning design and analytics, as well as a list of guidelines for technology development aiming to support IBL orchestration. These two contributions can support the creation of future learning analytics–enhanced IBL environments that are both pedagogically grounded and usable by teachers in authentic settings.
References
Alavi, H. S., & Dillenbourg, P. (2012). An ambient awareness tool for supporting supervised collaborative problem solving. IEEE Transactions on Learning Technologies, 5(3), 264–274. https://doi.org/10.1109/TLT.2012.7
Alfieri, L., Brooks, P. J., Aldrich, N. J., & Tenenbaum, H. R. (2011). Does discovery-based instruction enhance learning? Journal of Educational Psychology, 103(1), 1–18. https://doi.org/10.1037/a0021017
Bakharia, A., Corrin, L., de Barba, P., Kennedy, G., Gašević, D., Mulder, R., Williams, D., Dawson, S., & Lockyer, L. (2016). A conceptual framework linking learning design with learning analytics. Proceedings of the Sixth International Conference on Learning Analytics and Knowledge (LAK 2016), 25–29 April 2016, Edinburgh, UK, 329–338. https://doi.org/10.1145/2883851.2883944
Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Sciences, 13(1), 1–14. https://doi.org/10.1207/s15327809jls1301_1
Blake, C., Scanlon, E., Twiner, A., Collins, T., Jones, A., & Kerawalla, L. (2013). Analysis of learners’ fieldtrip talk during a collaborative inquiry task. Scaling up Learning for Sustained Impact, 32–42. https://doi.org/10.1007/978-3-642-40814-4_4
Blanchard, M. R., Southerland, S. A., & Granger, E. M. (2009). No silver bullet for inquiry: Making sense of teacher change following an inquiry-based research experience for teachers. Science Education, 93(2), 322–360. https://doi.org/10.1002/sce.20298
Blumenstein, M. (2020). Synergies of learning analytics and learning design: A systematic review of student outcomes. Journal of Learning Analytics, 7(3), 13–32. https://doi.org/10.18608/jla.2020.73.3
de Jong, T., Gillet, D., Rodr´ıguez-Triana, M. J., Hovardas, T., Dikke, D., Doran, R., Dziabenko, O., Koslowsky, J., Korventausta, M., Law, E., Pedaste, M., Tasiopoulou, E., Vidal, G., & Zacharia, Z. C. (2021). Understanding teacher design practices for digital inquiry–based science learning: The case of Go-Lab. Educational Technology Research and Development, 69, 417–444. https://doi.org/10.1007/s11423-020-09904-z
de Jong, T., Linn, M. C., & Zacharia, Z. C. (2013). Physical and virtual laboratories in science and engineering education. Science, 340(6130), 305–308. https://doi.org/10.1126/science.1230579
Dillenbourg, P., & Jermann, P. (2010). Technology for classroom orchestration. New Science of Learning (pp. 525–552). Springer. https://doi.org/10.1007/978-1-4419-5716-0_26
Dillenbourg, P., Zufferey, G., Alavi, H., Jermann, P., Do-Lenh, S., Bonnard, Q., Cuendet, S., & Kaplan, F. (2011). Classroom orchestration: The third circle of usability. In H. Spada, G. Stahl, N. Miyake, & N. Law (Eds.), Connecting Computer-Supported Collaborative Learning to Policy and Practice: CSCL2011 Conference Proceedings. Volume I—Long Papers 4–8 July 2011, Hong Kong (pp. 510–517). International Society of the Learning Sciences. https://repository.isls.org/handle/1/2491
Dimitrakopoulou, A., Petrou, A., Martinez, A., Marcos, J. A., Kollias, V., Jermann, P., Harrer, A., Dimitriadis, Y., & Bollen, L. (2006). State of the Art of Interaction Analysis for Metacognitive Support & Diagnosis (tech. rep. No. D.31.1). Kaleidoscope NoE. https://hal.archives-ouvertes.fr/hal-00190146/
Dobber, M., Zwart, R., Tanis, M., & van Oers, B. (2017). Literature review: The role of the teacher in inquiry-based education. Educational Research Review, 22, 194–214. https://doi.org/10.1016/j.edurev.2017.09.002
Dodero, J. M., Gonz´alez-Conejero, E. J., Guti´errez-Herrera, G., Peinado, S., Tocino, J. T., & Ruiz-Rube, I. (2017). Trade-off between interoperability and data collection performance when designing an architecture for learning analytics. Future Generation Computer Systems, 68, 31–37. https://doi.org/10.1016/j.future.2016.06.040
Dorier, J.-L., & Garc´ıa, F. J. (2013). Challenges and opportunities for the implementation of inquiry-based learning in day-to-day teaching. ZDM, 45(6), 837–849. https://doi.org/10.1007/s11858-013-0512-8
Emin-Martinez, V., Hansen, C., Rodr´ıguez-Triana, M. J., Wasson, B., Mor, Y., Dascalu, M., Ferguson, R., & Pernin, J.-P. (2014). Towards teacher-led design inquiry of learning. eLearning Papers, (36). https://infoscience.epfl.ch/record/234556
Friesel, A., Cojocaru, D., & Avramides, K. (2015). Identifying how PELARS-project can support the development of new curriculum structures in engineering education. Proceedings of the Third Experiment@ International Conference, 2–4 June 2015, Ponta Delgada, Azores, Portugal, 219–223. https://doi.org/10.1109/EXPAT.2015.7463269
Furtak, E. M., Seidel, T., Iverson, H., & Briggs, D. C. (2012). Experimental and quasi-experimental studies of inquiry-based science teaching: A meta-analysis. Review of Educational Research, 82(3), 300–329. https://doi.org/10.3102/0034654312457206
Gašević, D., Dawson, S., Rogers, T., & Gasevic, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education, 28, 68–84. https://doi.org/10.1016/j.iheduc.2015.10.002
Gillet, D., de Jong, T., Sotiriou, S., & Salzmann, C. (2013). Personalised learning spaces and federated online labs for STEM education at school: Supporting teacher communities and inquiry learning. Proceedings of the 15th IEEE Global Engineering Education Conference (EDUCON 2013), 13–15 March 2013, Berlin, Germany, 769–773. https://doi.org/10.1109/EduCon.2013.6530194
Gluga, R., Kay, J., Lister, R., Simon, Charleston, M., Harland, J., & Teague, D. (2013). A conceptual model for reflecting on expected learning vs. demonstrated student performance. Proceedings of the 15th Australasian Computing Education Conference (ACE 2013), 29 January–1 February 2013, Adelaide, Australia, 77–86. https://dl.acm.org/doi/10.5555/2667199.2667208
Guéraud, V., Lejeune, A., Adam, J.-M., Dubois, M., & Mandran, N. (2009). Supervising distant simulation-based practical work: Environment and experimentation. In U. Cress, V. Dimitrova, & M. Specht (Eds.), Learning in the Synergy of Multiple Disciplines (pp. 602–608). Springer. https://doi.org/10.1007/978-3-642-04636-0_55
Harris, C. J., & Rooks, D. L. (2010). Managing inquiry-based science: Challenges in enacting complex science instruction in elementary and middle school classrooms. Journal of Science Teacher Education, 21(2), 227–240. https://doi.org/10.1007/s10972-009-9172-5
Hernández-Leo, D., Martinez-Maldonado, R., Pardo, A., Mu˜noz-Crist´obal, J. A., & Rodr´ıguez-Triana, M. J. (2018). Analytics for learning design: A layered framework and tools: Analytics layers for learning design. British Journal of Educational Technology, 3(1), 139–152. https://doi.org/10.1111/bjet.12645
Hoel, T., & Chen, W. (2016). Privacy-driven design of learning analytics applications—Exploring the design space of solutions for data sharing and interoperability. Journal of Learning Analytics, 3(1), 139–158. https://doi.org/10.18608/jla.2016.31.9
Hofer, E., & Lembens, A. (2019). Putting inquiry-based learning into practice: How teachers changed their beliefs and attitudes through a professional development program. Chemistry Teacher International, 1(2), 20180030. https://doi.org/10.1515/cti-2018-0030
Holstein, K., McLaren, B. M., & Aleven, V. (2019). Designing for complementarity: Teacher and student needs for orchestration support in AI-enhanced classrooms. Proceedings of the 20th International Conference on Artificial Intelligence in Education (AIED 2019), 25–29 June 2019, Chicago, IL, USA, 157–171. https://doi.org/10.1007/978-3-030-23204-7_14
Hsu, Y.-S., Lai, T.-L., & Hsu, W.-H. (2015). A design model of distributed scaffolding for inquiry-based learning. Research in Science Education, 45(2), 241–273. https://doi.org/10.1007/s11165-014-9421-2
Jona, K., & Uttal, D. (2013). Don’t forget the teacher: New tools to support broader adoption of remote labs. Proceedings of the 10th International Conference on Remote Engineering and Virtual Instrumentation (REV 2013), 6–8 February 2013, Sydney, Australia. https://doi.org/10.1109/REV.2013.6502910
Kahneman, D., & Tversky, A. (1982). Intuitive prediction: Biases and corrective procedures. Judgment under Uncertainty: Heuristics and Biases (pp. 414–421). Cambridge University Press. https://doi.org/10.1017/CBO9780511809477.031
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75–86. https://doi.org/10.1207/s15326985ep4102_1
Kotler, P., & Armstrong, G. (2010). Principles of Marketing. Pearson Education.
Lejeune, A., Ney, M., Weinberger, A., Pedaste, M., Bollen, L., Hovardas, T., Hoppe, U., & de Jong, T. (2009). Learning activity spaces: Towards flexibility in learning design? Proceedings of the Ninth IEEE International Conference on Advanced Learning Technologies (ICALT 2009), 15–17 July 2009, Riga, Latvia, 433–437. https://doi.org/10.1109/ICALT.2009.100
Lockyer, L., & Dawson, S. (2011). Learning designs and learning analytics. Proceedings of the First International Conference on Learning Analytics and Knowledge (LAK 2011), 27 February–1 March 2011, Banff, AB, Canada, 153–156. https://doi.org/10.1145/2090116.2090140
Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing pedagogical action: Aligning learning analytics with learning design. American Behavioral Scientist, 57(10), 1439–1459. https://doi.org/10.1177/0002764213479367
Maaß, K., & Artigue, M. (2013). Implementation of inquiry-based learning in day-to-day teaching: A synthesis. ZDM, 45(6), 779–795. https://doi.org/10.1007/s11858-013-0528-0
Mangaroska, K., & Giannakos, M. N. (2018). Learning analytics for learning design: A systematic literature review of analytics-driven design to enhance learning. IEEE Transactions on Learning Technologies, 12(4), 516–534. https://doi.org/10.1109/TLT.2018.2868673
Manske, S., & Cao, Y. (2015). Specifications of the Learning Analytics, Scaffolding Services, and Add-on Services—Final (tech. rep. No. D4.6). Go-Lab. https://nextlab.golabz.eu/sites/project/files/inline-files/Go-Lab D4.6-ilovepdf-compressed.pdf
Manske, S., Hecking, T., Bollen, L., G¨ohnert, T., Ramos, A., & Hoppe, H. U. (2014). A flexible framework for the authoring of reusable and portable learning analytics gadgets. Proceedings of the 14th IEEE International Conference on Advanced Learning Technologies (ICALT 14), 7–9 July 2014, Athens, Greece, 254–258. https://doi.org/10.1109/ICALT.2014.80
Mulholland, P., Anastopoulou, S., Collins, T., Feisst, M., Gaved, M., Kerawalla, L., Paxton, M., Scanlon, E., Sharples, M., & Wright, M. (2012). nQuire: Technological support for personal inquiry learning. IEEE Transactions on Learning Technologies, 5(2), 157–169. https://doi.org/10.1109/TLT.2011.32
Ochoa, X., & Ternier, S. (2017). Technical learning infrastructure, interoperability and standards. In E. Duval, M. Sharples, & R. Sutherland (Eds.), Technology Enhanced Learning (pp. 145–155). Springer. https://doi.org/10.1007/978-3-319-02600-8_14
Pedaste, M., Lazonder, A., Raes, A., Wajeman, C., Moore, E., & Girault, I. (2015). Grand challenge problem 3: Empowering science teachers using technology-enhanced scaffolding to improve inquiry learning. SpringerBriefs in Education (pp. 17–20). Springer. https://doi.org/https://doi.org/10.1007/978-3-319-12562-6_4
Pedaste, M., Mäeots, M., Siiman, L. A., de Jong, T., van Riesen, S. A. N., Kamp, E. T., Manoli, C. C., Zacharia, Z. C., & Tsourlidaki, E. (2015). Phases of inquiry-based learning: Definitions and the inquiry cycle. Educational Research Review, 14, 47–61. https://doi.org/10.1016/j.edurev.2015.02.003
Prieto, L. P. (2015). Classroom technology design guidelines: A focused survey. Proceedings of the Orchestrated Collaborative Classroom Workshop, 7 June 2015, Gothenburg, Sweden, 10–14. http://ceur-ws.org/Vol-1411/paper-03.pdf
Prieto, L. P., Dimitriadis, Y., & Looi, C.-K. (2015). Orchestration in learning technology research: Evaluation of a conceptual framework. Research in Learning Technology, 23. https://doi.org/10.3402/rlt.v23.28019
Prieto, L. P., Holenko Dlab, M., Gutiérrez, I., Abdulwahed, M., & Balid, W. (2011). Orchestrating technology enhanced learning: a literature review and a conceptual framework. International Journal of Technology Enhanced Learning, 3(6), 583–598. https://doi.org/10.1504/IJTEL.2011.045449
Prieto, L. P., Rodríguez-Triana, M. J., Martínez-Maldonado, R., Dimitriadis, Y., & Gašević, D. (2019). Orchestrating Learning Analytics (OrLA): Supporting inter-stakeholder communication about adoption of learning analytics at the classroom level. Australasian Journal of Educational Technology, 35(4). https://doi.org/10.14742/ajet.4314
Quintana, C., Reiser, B. J., Davis, E. A., Krajcik, J., Fretz, E., Duncan, R. G., Kyza, E., Edelson, D., & Soloway, E. (2004). A scaffolding design framework for software to support science inquiry. Journal of the Learning Sciences, 13(3), 337–386. https://doi.org/10.1207/s15327809jls1303_4
Reimann, P. (2016). Connecting learning analytics with learning research: The role of design-based research. Learning: Research and Practice, 2(2), 130–142. https://doi.org/https://doi.org/10.1080/23735082.2016.1210198
Rienties, B., Toetenel, L., & Bryan, A. (2015). Scaling up learning design: Impact of learning design activities on LMS behaviour and performance. Proceedings of the Fifth International Conference on Learning Analytics and Knowledge (LAK 2015), 16–20 March 2015, Poughkeepsie, NY, USA, 315–319. https://doi.org/10.1145/2723576.2723600
Rodríguez-Triana, M. J., & Holzer, A. (2016). Release of a Sustainable Version of the Go-Lab Portal and the Associated End-User Solutions (tech. rep. No. D5.7). Go-Lab. https://nextlab.golabz.eu/sites/project/files/inline- files/Go-Lab_D5.7-ilovepdf-compressed.pdf
Rodríguez-Triana, M. J., Martínez-Monés, A., Asensio-Pérez, J. I., & Dimitriadis, Y. (2015). Scripting and monitoring meet each other: Aligning learning analytics and learning design to support teachers in orchestrating CSCL situations. British Journal of Educational Technology, 46(2), 330–343. https://doi.org/10.1111/bjet.12198
Rodríguez-Triana, M. J., Prieto, L. P., Ley, T., de Jong, T., & Gillet, D. (2020). Social practices in teacher knowledge creation and innovation adoption: A large-scale study in an online instructional design community for inquiry learning. International Journal of Computer-Supported Collaborative Learning, 15(4), 445–467. https://doi.org/10.1007/s11412-020-09331-5
Rodríguez-Triana, M. J., Prieto, L. P., Martínez-Monés, A., Asensio-Pérez, J. I., & Dimitriadis, Y. (2018). The teacher in the loop: Customizing multimodal learning analytics for blended learning. Proceedings of the Eighth International Conference on Learning Analytics and Knowledge (LAK 2018), 5–9 March 2018, Sydney, Australia, 417–426. https://doi.org/10.1145/3170358.3170364
Rodr´ıguez-Triana, M. J., Prieto Santos, L. P., Vozniuk, A., Shirvani Boroujeni, M., Schwendimann, B. A., Holzer, A. C., & Gillet, D. (2017). Monitoring, awareness and reflection in blended technology enhanced learning: A systematic review. International Journal of Technology Enhanced Learning, 9(2/3), 126–150. https://doi.org/10.1504/IJTEL.2017.084489
Scheffel, M., Drachsler, H., Toisoul, C., Ternier, S., & Specht, M. (2017). The proof of the pudding: Examining validity and reliability of the evaluation framework for learning analytics. In É. Lavoué, H. Drachsler, K. Verbert, J. Broisin, & M. Pérez-Sanagustín (Eds.), Data Driven Approaches in Digital Education (pp. 194–208). Springer International Publishing. https://doi.org/10.1007/978-3-319-66610-5_15
Schwendimann, B. A., Rodr´ıguez-Triana, M. J., Vozniuk, A., Prieto, L. P., Boroujeni, M. S., Holzer, A., Gillet, D., & Dillenbourg, P. (2016). Perceiving learning at a glance: A systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies, 10(1), 30–41. https://doi.org/10.1109/TLT.2016.2599522
Sergis, S., Sampson, D. G., Rodr´ıguez-Triana, M. J., Gillet, D., Pelliccione, L., & de Jong, T. (2019). Using educational data from teaching and learning to inform teachers’ reflective educational design in inquiry-based STEM education. Computers in Human Behavior, 92, 724–738. https://doi.org/10.1016/j.chb.2017.12.014
Seymour, E., Hunter, A.-B., Laursen, S. L., & DeAntoni, T. (2004). Establishing the benefits of research experiences for undergraduates in the sciences: First findings from a three-year study. Science Education, 88(4), 493–534. https://doi.org/10.1002/sce.10131
Sharples, M. (2013). Shared orchestration within and beyond the classroom. Computers & Education, 69, 504–506. https://doi.org/10.1016/j.compedu.2013.04.014
Sharples, M., & Anastopoulou, S. (2012). Designing orchestration for inquiry learning. In K. Littleton, E. Scanlon, & M. Sharples (Eds.), Orchestrating Inquiry Learning (pp. 69–85). Routledge. http://www.routledge.com/books/details/9780415601139/
Shen, H., Liang, L., Law, N., Hemberg, E., & O’Reilly, U.-M. (2020). Understanding learner behavior through learning design informed learning analytics. Proceedings of the Seventh ACM Conference on Learning @ Scale (L@S 2020), 12–20 August 2020, online, 135–145. https://doi.org/10.1145/3386527.3405919
Silm, G., Tiitsaar, K., Pedaste, M., Zacharia, Z. C., & Papaevripidou, M. (2017). Teachers’ readiness to use inquiry-based learning: An investigation of teachers’ sense of efficacy and attitudes toward inquiry-based learning. Science Education International, 28(4), 315–325. https://eric.ed.gov/?id=EJ1161535
Slotta, J. D., Tissenbaum, M., & Lui, M. (2013). Orchestrating of complex inquiry: Three roles for learning analytics in a smart classroom infrastructure. Proceedings of the Third International Conference on Learning Analytics and Knowledge (LAK 2013), 8–12 April 2013, Leuven, Belgium, 270–274. https://doi.org/10.1145/2460296.2460352
Voet, M., & De Wever, B. (2019). Teachers’ adoption of inquiry-based learning activities: The importance of beliefs about education, the self, and the context. Journal of Teacher Education, 70(5), 423–440. https://doi.org/10.1177/0022487117751399
Vozniuk, A., Rodríguez-Triana, M. J., Holzer, A., Govaerts, S., Sandoz, D., & Gillet, D. (2015). Contextual learning analytics apps to create awareness in blended inquiry learning. Proceedings of the 2015 International Conference on Information Technology Based Higher Education and Training (ITHET 2015), 11–13 June 2015, Lisbon, Portugal, 1–5. https://doi.org/10.1109/ITHET.2015.7218029
Wang, F., & Hannafin, M. J. (2005). Design-based research and technology-enhanced learning environments. Educational Technology, Research and Development, 53(4), 5–23. https://doi.org/10.1007/BF02504682
Wiley, K. J., Dimitriadis, Y., Bradford, A., & Linn, M. C. (2020). From theory to action: Developing and evaluating learning analytics for learning design. Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK 2020), 23–27 March 2020, Frankfurt, Germany, 569–578. https://doi.org/10.1145/3375462.3375540
Zacharia, Z. C., Manoli, C., Xenofontos, N., De Jong, T., Pedaste, M., van Riesen, S. A., Kamp, E. T., Mäeots, M., Siiman, L., & Tsourlidaki, E. (2015). Identifying potential types of guidance for supporting student inquiry when using virtual and remote labs in science: A literature review. Educational Technology Research and Development, 63(2), 257–302. https://doi.org/10.1007/s11423-015-9370-0
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2021 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
