Using Innovative Methods to Explore the Potential of an Alerting Dashboard for Science Inquiry
DOI:
https://doi.org/10.18608/jla.2021.7153Keywords:
teacher dashboard, epistemic network analyses, science inquiryAbstract
Educational technologies, such as teacher dashboards, are being developed to support teachers’ instruction and students’ learning. Specifically, dashboards support teachers in providing the just-in-time instruction needed by students in complex contexts such as science inquiry. In this study, we used the Inq-Blotter teacher-alerting dashboard to investigate whether teacher support elicited by the technology influenced students’ inquiry performance in a science intelligent tutoring system, Inq-ITS. Results indicated that students’ inquiry improved after receiving teachers’ help, elicited by the Inq-Blotter alerts. This inquiry improvement was significantly greater than for matched students who did not receive help from the teacher in response to alerts. Epistemic network analyses were then used to investigate the patterns in the discursive supports provided to students by teachers. These analyses revealed significant differences in the types of support that fostered (versus did not foster) student improvement; differences across teachers were also found. Overall, this study used innovative tools and analyses to understand how teachers use this technological genre of alerting dashboards to dynamically support students in science inquiry.
References
Acosta, A., & Slotta, J. D. (2018). CKBiology: An active learning curriculum design for secondary biology. Frontiers in Education, 3, 1–19. https://doi.org/10.3389/feduc.2018.00052
Adair, A., Dickler, R., & Gobert, J. (2020). Supporting teachers supporting students: Evidence-based TIPS in a dashboard to guide inquiry scaffolding. In Proceedings of the 14th International Conference of the Learning Sciences (ICLS 2020), 19–23 June 2020, Nashville, TN, USA (pp. 1769–1770). ISLS. Retrieved from https://repository.isls.org/bitstream/1/6434/1/1769-1770.pdf
Aleven, V., Roll, I., McLaren, B. M., & Koedinger, K. R. (2016). Help helps, but only so much: Research on help seeking with intelligent tutoring systems. International Journal of Artificial Intelligence in Education, 26(1), 205–223. https://doi.org/10.1007/s40593-015-0089-1
Charleer, S., Klerkx, J., & Duval, E. (2014). Learning dashboards. Journal of Learning Analytics, 1(3), 199–202. https://doi.org/10.18608/jla.2014.13.22
Dickler, R., Gobert, J., & Sao Pedro, M. (2019a). Using epistemic network analysis to characterize teacher discourse in response to an alerting dashboard. Paper presented at the 29th Annual Meeting of the Society for Text and Discourse (ST&D 2019), 9–11 July 2019, New York.
Dickler, R., Li, H., & Gobert, J. (2019b). Teacher scaffolds mediated by a science inquiry dashboard. Paper presented at the 14th Conference of the European Science Education Research Association (ESERA 2019), 26–30 August 2019, Bologna, Italy.
Dillenbourg, P. (2013). Design for classroom orchestration. Computers & Education, 69, 485–492. https://doi.org/10.1016/j.compedu.2013.04.013
Gee, J. P. (2004). An Introduction to discourse analysis: Theory and method. London, UK: Routledge. https://doi.org/10.4324/9780203005675
Gobert, J. D., Baker, R. S., & Sao Pedro, M. A. (2014). Inquiry skills tutoring system. U.S. Patent No. 9,373,082. Washington, DC: U.S. Patent and Trademark Office.
Gobert, J., Moussavi, R., Li, H., Sao Pedro, M., & Dickler, R. (2018). Real-time scaffolding of students’ online data interpretation during inquiry with Inq-ITS using educational data mining. In A. K. M. Azad, M. Auer, A. Edwards, & T. de Jong (Eds.), Cyber-physical laboratories in engineering and science education (pp. 191–217). Cham: Springer. https://doi.org/10.1007/978-3-319-76935-6_8
Gobert, J. D., & Sao Pedro, M. A. (2017). Digital assessment environments for scientific inquiry practices. In A. A. Rupp & J. P. Leighton (Eds.), The Wiley handbook of cognition and assessment: Frameworks, methodologies, and applications (pp. 508–534), West Sussex, UK: Wiley. https://doi.org/10.1002/9781118956588.ch21
Gobert, J., Sao Pedro, M., Betts, C., & Baker, R. S. (2016). Inquiry skills tutoring system (child patent for alerting system). U.S. Patent No. 9,564,057. Washington, DC: U.S. Patent and Trademark Office.
Gobert, J. D., Sao Pedro, M., Raziuddin, J., & Baker, R. S. (2013). From log files to assessment metrics: Measuring students’ science inquiry skills using educational data mining. Journal of the Learning Sciences, 22(4), 521–563. https://doi.org/10.1080/10508406.2013.837391
Holstein, K., McLaren, B. M., & Aleven, V. (2018a). Student learning benefits of a mixed-reality teacher awareness tool in AI-enhanced classrooms. In Proceedings of the 19th International Conference on Artificial Intelligence in Education (AIED 2018), 27–30 June 2018, London, UK (pp. 154–168). Cham: Springer. https://doi.org/10.1007/978-3-319-93843-1_12
Holstein, K., McLaren, B. M., & Aleven, V. (2018b). Informing the design of teacher awareness tools through causal alignment analysis. In Proceedings of the 13th International Conference of the Learning Sciences (ICLS 2018), 23–27 June 2018, London, UK (pp. 104–111).
Holstein, K., McLaren, B. M., & Aleven, V. (2019). Co-designing a real-time classroom orchestration tool to support teacher–AI complementarity. Journal of Learning Analytics, 6(2), 27–52. https://doi.org/10.18608/jla.2019.62.3
Howe, C., Hennessy, S., Mercer, N., Vrikki, M., & Wheatley, L. (2019). Teacher–student dialogue during classroom teaching: Does it really impact on student outcomes? Journal of the Learning Sciences, 28(4-5), 462–512. https://doi.org/10.1080/10508406.2019.1573730
Jurow, A. S., & Creighton, L. (2005). Improvisational science discourse: Teaching science in two K-1 classrooms. Linguistics and Education, 16(3), 275–297. https://doi.org/10.1016/j.linged.2006.02.002
Laurillard, D. (2008). The teacher as action researcher: Using technology to capture pedagogic form. Studies in Higher Education, 33(2), 139–154. https://doi.org/10.1080/03075070801915908
Manz, E., & Renga, I. P. (2017). Understanding how teachers guide evidence construction conversations. Science Education, 101(4), 584–615. https://doi.org/10.1002/sce.21282
Marquart, C. L., Hinojosa, C., Swiecki, Z., Eagan, B., & Shaffer, D. W. (2018). Epistemic network analysis (version 1.5.2). Retrieved from https://www.epistemicnetwork.org/
Martinez-Maldonado, R., Clayphan, A., Yacef, K., & Kay, J. (2015). MTFeedback: Providing notifications to enhance teacher awareness of small group work in the classroom. IEEE Transactions on Learning Technologies, 8(2), 187–200. https://doi.org/10.1109/TLT.2014.2365027
Martinez-Maldonado, R., Kay, J., Yacef, K., Edbauer, M. T., & Dimitriadis, Y. (2013). MTClassroom and MTDashboard: Supporting analysis of teacher attention in an orchestrated multi-tabletop classroom. In Proceedings of the 10th International Conference on Computer Supported Collaborative Learning (CSCL 2013), 15–19 June 2013, Madison, WI, USA. (pp. 320–327).
Martinez-Maldonado, R., Kay, J., Yacef, K., & Schwendimann, B. (2012). An interactive teacher’s dashboard for monitoring groups in a multi-tabletop learning environment. In Proceedings of the International Conference on Intelligent Tutoring Systems (ITS 2012), 14–18 June 2012, Chania, Greece (pp. 482–492). Springer. https://doi.org/10.1007/978-3-642-30950-2_62
Matuk, C., Gerard, L., Lim-Breitbart, J., & Linn, M. (2016). Gathering requirements for teacher tools: Strategies for empowering teachers through co-design. Journal of Science Teacher Education, 27(1), 79–110. https://doi.org/10.1007/s10972-016-9459-2
Mislevy, R. J., Behrens, J. T., Dicerbo, K. E., & Levy, R. (2012). Design and discovery in educational assessment: Evidence-centered design, psychometrics, and educational data mining. Journal of Educational Data Mining, 4(1), 11–48. https://doi.org/10.5281/zenodo.3554641
Molenaar, I., & Knoop-van Campen, C. (2017). Teacher dashboards in practice: Usage and impact. In Proceedings of the 12th European Conference on Technology Enhanced Learning (EC-TEL 2017), 12–15 September 2017, Tallinn, Estonia (pp. 125–138). Springer, Cham. https://doi.org/10.1007/978-3-319-66610-5_10
Molenaar, I., &. Knoop-van Campen, C. (2018). Experience matters: The impact of dashboards on teachers’ feedback. In Proceedings of the 13th International Conference of the Learning Sciences (ICLS ’18), 23–27 June 2018, London, UK (pp. 1231–1232).
Next Generation Science Standards Lead States. (2013). Next generation science standards: For states, by states. The National Academies Press. https://doi.org/10.17226/18290
Pruitt, S. L. (2014). The next generation science standards: The features and challenges. Journal of Science Teacher Education, 25(2), 145–156. https://doi.org/10.1007/s10972-014-9385-0
Roschelle, J., Martin, W., Ahn, J., & Schank, P. (Eds.) (2017). Cyberlearning community report: The state of cyberlearning and the future of learning with technology (technical report). SRI International. Retrieved from http://circlcenter.org/wp-content/uploads/2017/07/CyberlearningCommunityReport2017.pdf
Rupp, A. A., Choi, Y., Gushta, M., Mislevy, R., Thies, M.C., Bagley, E., … Shaffer, D. (2009). Modeling learning progressions in epistemic games with epistemic network analysis: Principles for data analysis and generation. In Proceedings of the 2009 Learning Progressions in Science Conference (LeaPS 2009), 24–26 June 2009, Iowa City, IA, USA (pp. 24–26). Retrieved from https://www.education.msu.edu/projects/leaps/proceedings/Rupp.pdf
Sao Pedro, M. A., Gobert, J., & Dickler, R. (2019). Can an alerting teacher dashboard improve how teachers help their students learn science inquiry practices? Paper presented at the 2019 American Educational Research Association Annual Meeting: Learning and Instruction (AERA 2019), 5–9 April 2019, Toronto, ON, Canada.
Schwarz, B. B., Prusak, N., Swidan, O., Livny, A., Gal, K., & Segal, A. (2018). Orchestrating the emergence of conceptual learning: A case study in a geometry class. International Journal of Computer-Supported Collaborative Learning, 13(2), 189–211. https://doi.org/10.1007/s11412-018-9276-z
Segal, A., Hindi, S., Prusak, N., Swidan, O., Livni, A., Palatnic, A., & Schwarz, B. (2017). Keeping the teacher in the loop: Technologies for monitoring group learning in real-time. In Proceedings of the 18th International Conference on Artificial Intelligence in Education (AIED 2017), 28 June–1 July 2017, Wuhan, China (pp. 64–76). Cham: Springer. https://doi.org/10.1007/978-3-319-61425-0_6
Shaffer, D. W., Collier, W., & Ruis, A. R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3(3), 9–45. https://doi.org/10.18608/jla.2016.33.3
Sherin, M., Jacobs, V., & Philipp, R. (Eds.). (2011). Mathematics teacher noticing: Seeing through teachers’ eyes. London, UK: Routledge.
Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 7, 1–22. https://doi.org/10.17763/haer.57.1.j463w79r56455411
Svarovsky, G. N. (2011). Exploring complex engineering learning over time with epistemic network analysis. Journal of Pre-College Engineering Education, 1(2), Article 4. https://doi.org/10.5703/1288284314638
Tissenbaum, M., & Slotta, J. (2019). Supporting classroom orchestration with real-time feedback: A role for teacher dashboards and real-time agents. International Journal of Computer-Supported Collaborative Learning, 14(3), 325–351. https://doi.org/10.1007/s11412-019-09306-1
Tabak, I., & Kyza, E. A. (2018). Research on scaffolding in the learning sciences: A methodological perspective. In F. Fischer, C. E. Hmelo-Silver, S. R. Goldman, & Peter Reimann (Eds.), International handbook of the learning sciences (pp. 191–200). Routledge. https://doi.org/10.4324/9781315617572-19
van Leeuwen, A. & Rummel, N. (2018). Considering teachers’ informational needs and the role division between teacher and orchestration tool in the context of collaborative learning. In Proceedings of the 13th International Conference of the Learning Sciences (ICLS 2018), 23–27 June 2018, London, UK (pp. 1230–1231).
van Leeuwen, A., van Wermeskerken, M., Erkens, G., & Rummel, N. (2017). Measuring teacher sense making strategies of learning analytics: A case study. Learning: Research and Practice, 3(1), 42–58. https://doi.org/10.1080/23735082.2017.1284252
Verbert, K., Govaerts, S., Duval, E., Santos, J. L., Van Assche, F., Parra, G., & Klerkx, J. (2014). Learning dashboards: An overview and future research opportunities. Personal and Ubiquitous Computing, 18(6), 1499–1514. https://doi.org/10.1007/s00779-013-0751-2
West, D. M. (2012). Big data for education: Data mining, data analytics, and web dashboards. Governance Studies at Brookings. Retrieved from https://www.brookings.edu/wp-content/uploads/2016/06/04-education-technology-west.pdf
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2021 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
