A Multilevel Framework and Method for Learning Analytics Integrated Learning Design
DOI:
https://doi.org/10.18608/jla.2020.73.8Keywords:
Multilevel learning design taxonomy, learning analytics taxonomy, inquiry-oriented learning design, integrated LD-LA framework, Learning Analytics integrated Curriculum Component Design Patterns (LA-CCDP)Abstract
Efforts to realize the potential of learning analytics (LA) to contribute to improving student learning and learning design have brought important advances. A review of successful cases of learning analytics applications reveals that 1) there is a tight coupling between the learning outcome (LO) goals, task sequence design, and the learning analytics and feedback in each case, and 2) the learning analytics to be deployed and the feedback to be provided to learners and/or teachers are integral to the learning design (LD) rather than constructed after the LD is completed. Learning design frameworks in the literature have focused on generic learning task taxonomies and are unable to scaffold LA-integrated LD practice. This paper proposes a multilevel framework for LA-integrated LD, which provides a hierarchically nested multilevel structure for the design of LD and LA elements based on 60 STEM curriculum units collected from authentic classrooms. The framework includes a design process model in the form of a Learning Design Triangle and the concept of Learning Analytics integrated Curriculum Component Design Patterns (LA-CCDP). Operationalization of the framework is illustrated using one STEM curriculum unit. This framework can be adopted for professional learning and technology development to support LA-integrated LD practices.
References
Ahn, J., Campos, F., Hays, M., & DiGiacomo, D. (2019). Designing in context: Reaching beyond usability in learning analytics dashboard design. Journal of Learning Analytics, 6(2), 70–85. https://dx.doi.org/10.18608/jla.2019.62.5
Alexander, C. (1977). A pattern language: Towns, buildings, construction. Oxford, UK: Oxford University Press.
Barrows, H. S. (1986). A taxonomy of problem-based learning methods. Medical Education, 20(6), 481–486. https://dx.doi.org/10.1111/j.1365-2923.1986.tb01386.x
Bower, M., Craft, B., Laurillard, D., & Masterman, L. (2011). Using the learning designer to develop a conceptual framework for linking learning design tools and system. In L. Cameron & J. Dalziel (Eds.), Proceedings of the 6th International LAMS & Learning Design Conference: Learning Design for a Changing World, 8–9 December 2011, Sydney, Australia (pp. 61–71). Sydney, Australia: LAMS Foundation. Retrieved from https://researchers.mq.edu.au/en/publications/using-the-learning-designer-to-develop-a-conceptual-framework-for
Boyle, T. (2010). Layered learning design: Towards an integration of learning design and learning object perspectives. Computers & Education, 54(3), 661–668. https://dx.doi.org/10.1016/j.compedu.2009.09.026
Buckingham Shum, S., Ferguson, R., & Martinez-Maldonado, R. (2019). Human-centred learning analytics. Journal of Learning Analytics, 6(2), 1–9. https://dx.doi.org/10.18608/jla.2019.62.1
Charlton, P., Magoulas, G., & Laurillard, D. (2012). Enabling creative learning design through semantic technologies. Technology, Pedagogy and Education, 21(2), 231–253. https://dx.doi.org/10.1080/1475939x.2012.698165
Corrin, L., Kennedy, G., de Barba, P. G., Lockyer, L., Gašević, D., Williams, D., & Bakharia, A. (2016). Completing the loop: Returning meaningful learning analytic data to teachers. Sydney: Office for Learning and Teaching. Retrieved from http://melbourne-cshe.unimelb.edu.au/__data/assets/pdf_file/0006/2083938/Loop_Handbook.pdf
Cross, S., Galley, R., Brasher, A., & Weller, M. (2012). The final report of the OULDI-JISC project: Challenge and change in curriculum design process. Bristol, UK: JISC.
Dalziel, J. (2013). Implementing learning design: A decade of lessons learned. Proceedings of the 30th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE 2013) 1–4 December 2013, Sydney, Australia (pp. 210–220). Australasian Society for Computers in Learning in Tertiary Education. Retrieved from https://www.ascilite.org/conferences/sydney13/program/papers/Dalziel,%20James.pdf
Earl, L. M. (2012). Assessment as learning: Using classroom assessment to maximize student learning. Thousand Oaks, CA: Corwin Press.
Gibson, A., Kitto, K., & Willis, J. (2014). A cognitive processing framework for learning analytics. Proceedings of the 4th International Conference on Learning Analytics and Knowledge (LAK ʼ14), 24–28 March 2014, Indianapolis, IN, USA (pp. 212–216). New York, NY: ACM. https://dx.doi.org/10.1145/2567574.2567610
Goodyear, P. (2005). Educational design and networked learning: Patterns, pattern languages and design practice. Australasian Journal of Educational Technology, 21(1), 82–101. https://dx.doi.org/10.14742/ajet.1344
Goodyear, P., & Retalis, S. (2010). Technology-enhanced learning: Design patterns and pattern languages. Rotterdam, Netherlands: Sense Publishers.
Gruber, T. R. (1995). Toward principles for the design of ontologies used for knowledge sharing? International Journal of Human-Computer Studies, 43(5–6), 907–928. https://dx.doi.org/10.1006/ijhc.1995.1081
Haya, P. A., Daems, O., Malzahn, N., Castellanos, J., & Hoppe, H. U. (2015). Analysing content and patterns of interaction for improving the learning design of networked learning environments. British Journal of Educational Technology, 46(2), 300–316.
Hernández-Leo, D., Harrer, A., Dodero, J. M., Asensio-Pérez, J. I., & Burgos, D. (2007). A framework for the conceptualization of approaches to “Create-by-Reuse” of learning design solutions. Journal of Universal Computer Science, 13(7), 991–1001. Retrieved from https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.114.1937&rep=rep1&type=pdf
Hew, K. F., Law, N., Wan, J., Lee, Y., & Kwok, A. (2016). Self-directed learning in science education: Explicating the enabling factors. Singapore: International Society of the Learning Sciences.
Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn? Educational Psychology Review, 16(3), 235–266. https://dx.doi.org/10.1023/b:edpr.0000034022.16470.f3
Inventado, P. S., & Scupelli, P. (2015). Data-driven design pattern production: A case study on the ASSISTments online learning system. Proceedings of the 20th European Conference on Pattern Languages of Programs (EuroPLoP ’15), 8–12 July 2015, Kaufbeuren, Germany (Article No. 14). New York, NY: ACM. https://dx.doi.org/10.1145/2855321.2855336
Koedinger, K. R., Stamper, J. C., McLaughlin, E. A., & Nixon, T. (2013). Using data-driven discovery of better student models to improve student learning. In H. C. Lane, K. Yacef, J. Mostow, & P. Pavlik (Eds.), Proceedings of the 16th International Conference on Artificial Intelligence in Education (AIED ʼ13), 9–13 July 2013, Memphis, TN, USA (pp. 421–430). Berlin, Germany: Springer.
Koehler, M., & Mishra, P. (2009). What is technological pedagogical content knowledge (TPACK)? Contemporary Issues in Technology and Teacher Education, 9(1), 60–70.
Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice, 41(4), 212–218. https://dx.doi.org/10.1207/s15430421tip4104_2
Laurillard, D. (2013). Teaching as a design science: Building pedagogical patterns for learning and technology. New York, NY: Routledge.
Law, N., Li, L., Farias Herrera, L., Chan, A., & Pong, T. C. (2017). A pattern language based learning design studio for an analytics informed inter-professional design community. Interaction Design and Architecture(s), 33, 92–112.
Liu, R., & Koedinger, K. R. (2017). Closing the loop: Automated data-driven cognitive model discoveries lead to improved instruction and learning gains. Journal of Educational Data Mining, 9(1), 25–41. https://dx.doi.org/10.5281/zenodo.3554625
Lockyer, L., & Dawson, S. (2012). Where learning analytics meets learning design. In S. Buckingham Shum, D. Gašević, & R. Ferguson (Eds.), Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK ʼ12), 29 April–2 May 2012, Vancouver, BC, Canada (pp. 14–15). New York, NY: ACM. https://dx.doi.org/10.1145/2330601.2330609
Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing pedagogical action: Aligning learning analytics with learning design. American Behavioral Scientist, 57(10), 1439–1459. https://dx.doi.org/10.1177/0002764213479367
Mangaroska, K., & Giannakos, M. N. (2019). Learning analytics for learning design: A systematic literature review of analytics-driven design to enhance learning. IEEE Transactions on Learning Technologies, 12(4), 516–534. https://dx.doi.org/10.1109/TLT.2018.2868673
Mavrikis, M., Geraniou, E., Gutierrez Santos, S., & Poulovassilis, A. (2019). Intelligent analysis and data visualization for teacher assistance tools: The case of exploratory learning. British Journal of Educational Technology, 50(6), 2920–2942. https://dx.doi.org/10.1111/bjet.12876
Mor, Y., Mellar, H., Warburton, S., & Winters, N. (2014). Practical design patterns for teaching and learning with technology. Rotterdam, The Netherlands: Sense Publishers. https://dx.doi.org/10.1007/978-94-6209-530-4
Nguyen, A., Gardner, L. A., & Sheridan, D. (2017). A multilayered taxonomy of learning analytics applications. Proceedings of the 21st Pacific-Asia Conference on Information Systems (PACIS 2017), 16–20 July 2017, Langkawi Island, Malaysia (Article No. 54). Atlanta, Georgia: Association for Information Systems (AIS). Retrieved from https://aisel.aisnet.org/pacis2017/54
Pilling-Cormick, J., & Garrison, D. R. (2007). Self-directed and self-regulated learning: Conceptual links. Canadian Journal of University Continuing Education, 33(2), 13–33. https://dx.doi.org/10.21225/D5S01M
Peña-Ayala, A. (2018). Learning analytics: A glance of evolution, status, and trends according to a proposed taxonomy. Data Mining and Knowledge Discovery, 8(3), e1243. https://dx.doi.org/10.1002/widm.1243
Prieto, L. P., Rodríguez-Triana, M. J., Martínez-Maldonado, R., Dimitriadis, Y., & Gašević, D. (2019). Orchestrating learning analytics (OrLA): Supporting inter-stakeholder communication about adoption of learning analytics at the classroom level. Australasian Journal of Educational Technology, 35(4), 14–33. https://dx.doi.org/https://doi.org/10.14742/ajet.4314
Rehrey, G., Shepard, L., Hostetter, C., Reynolds, A., & Groth, D. (2019). Engaging faculty in learning analytics: Agents of institutional culture change. Journal of Learning Analytics, 6(2), 86–94. http://dx.doi.org/10.18608/jla.2019.62.6
Rienties, B., Nguyen, Q., Holmes, W., & Reedy, K. (2017). A review of ten years of implementation and research in aligning learning design with learning analytics at the Open University UK. Interaction Design and Architecture(s), 33, 134–154.
Rienties, B., & Toetenel, L. (2016). The impact of learning design on student behaviour, satisfaction and performance: A cross-institutional comparison across 151 modules. Computers in Human Behavior, 60, 333–341. https://dx.doi.org/10.1016/j.chb.2016.02.074
Rienties, B., Toetenel, L., & Bryan, A. (2015). Scaling up learning design: Impact of learning design activities on LMS behavior and performance. Proceedings of the 5th International Conference on Learning Analytics and Knowledge (LAK ʼ15), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 315–319). New York, NY: ACM. http://dx.doi.org/doi:10.1145/2723576.2723600
Rosé, C. P., McLaughlin, E. A., Liu, R., & Koedinger, K. R. (2019). Explanatory learner models: Why machine learning (alone) is not the answer. British Journal of Educational Technology, 50(6), 2943–2958. http://dx.doi.org/10.1111/bjet.12858
Seufert, S., Meier, C., Soellner, M., & Rietsche, R. (2019). A pedagogical perspective on big data and learning analytics: A conceptual model for digital learning support. Technology, Knowledge and Learning, 24(4), 599–619. https://dx.doi.org/10.1007/s10758-019-09399-5
Shibani, A., Knight, S., Buckingham Shum, S., & Ryan, P. (2017). Design and implementation of a pedagogic intervention using writing analytics. In W. Chen et al. (Eds.), Proceedings of the 25th International Conference on Computers in Education (ICCE 2017), 4–8 December 2017, Christchurch, New Zealand (pp. 306–315). Jhongli City, Taiwan: Asia-Pacific Society for Computers in Education.
Shibani, A., Knight, S., & Shum, S. B. (2019). Contextualizable learning analytics design: A generic model and writing analytics evaluations. Proceedings of the 9th International Conference on Learning Analytics and Knowledge (LAK ’19), 4–8 March 2019, Tempe, Arizona, USA (pp. 210–219). New York, NY: ACM. https://dx.doi.org/10.1145/3303772.3303785
Shuler, C. F. (2012). Comparisons in basic science learning outcomes between students in PBL and traditional dental curricula at the same dental school. In S. Bridges, C. McGrath & T. Whitehill (Eds.), Problem-Based Learning in Clinical Education (pp. 35–46). Dordrecht, The Netherlands: Springer. http://dx.doi.org/10.1007/978-94-007-2515-7_3
Spady, W. (1993). Outcome-based education. Belconnen, ACT: Australian Curriculum Studies Association.
Toetenel, L., & Rienties, B. (2016). Learning design: Creative design to visualise learning activities. Open Learning: The Journal of Open, Distance and e-learning, 31(3), 233–244. http://dx.doi.org/10.1080/02680513.2016.1213626
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2020 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
