A Mixed-Method Study of How Instructors Design for Learning in Online and Distance Education
DOI:
https://doi.org/10.18608/jla.2020.73.6Keywords:
Learning design, Learning analytics, Mixed-methodAbstract
The use of analytical methods from learning analytics (LA) research combined with visualizations of learning activities using learning design (LD) tools and frameworks has provided important insight into how instructors design for learning. Nonetheless, there are many subtle nuances in instructors’ design decisions that might not easily be captured using LA tools. Therefore, this study sets out to explore how and why instructors design for learning in an online and distance higher education setting by employing a mixed-method approach, which combined semi-structured interviews of 12 instructors with network analyses of their LDs. Our findings uncovered several underlying factors that influenced how instructors designed their modules and highlighted some discrepancies between instructors’ pedagogical beliefs and their actual LD as captured by the Open University Learning Design Initiative (OULDI). This study showcases the potential of combining LA with qualitative insights for a better understanding of the complex design process in online distance higher education.
References
AUTCLearningDesign. (2002). Predict–observe–explain: Designer’s voice — Context. Retrieved from http://www.learningdesigns.uow.edu.au/exemplars/info/LD44/more/03Context.html
Bakharia, A., Corrin, L., de Barba, P., Kennedy, G., Gašević, D., Mulder, R., …, Lockyer, L. (2016). A conceptual framework linking learning design with learning analytics. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼ16), 25–29 April 2016, Edinburgh, UK, (pp. 329–338). New York, NY: ACM. https://dx.doi.org/10.1145/2883851.2883944
Bennett, S., Agostinho, S., & Lockyer, L. (2015). Technology tools to support learning design: Implications derived from an investigation of university teachers’ design practices. Computers & Education, 81, 211–220. https://dx.doi.org/10.1016/j.compedu.2014.10.016
Borgatti, S. P., Everett, M. G., & Freeman, L. C. (2002). Ucinet for Windows: Software for social network analysis. Harvard, MA: Analytic Technologies. Retrieved from https://sites.google.com/site/ucinetsoftware/home
Braun, V., & Clarke, V. (2012). Thematic analysis. In H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological. (pp. 57–71). Washington, D.C.: American Psychological Association. https://dx.doi.org/10.1037/13620-000
Chambers, E. (1992). Work-load and the quality of student learning. Studies in Higher Education, 17(2), 141–153. https://dx.doi.org/10.1080/03075079212331382627
Cherney, M. R., Fetherston, M., & Johnsen, L. J. (2017). Online course student collaboration literature: A review and critique. Small Group Research, 49(1), 98–128. https://dx.doi.org/10.1177/1046496417721627
Conole, G. (2012). Designing for learning in an open world (Vol. 4). New York, NY: Springer Science & Business Media. https://dx.doi.org/10.1007/978-1-4419-8517-0
Creswell, J. W., & Poth, C. N. (2017). Qualitative inquiry and research design: Choosing among five approaches. London, UK: Sage Publications. https://us.sagepub.com/en-us/nam/qualitative-inquiry-and-research-design/book246896
Cross, S., Galley, R., Brasher, A., & Weller, M. (2012). Final project report of the OULDI-JISC Project: Challenge and change in curriculum design process, communities, visualisation and practice. York, UK: JISC. Retrieved from http://www.jisc.ac.uk/media/documents/programmes/curriculumdesign/OULDI_Final_Report_instit%20story.pdf
Dalziel, J. (2003). Implementing learning design: The learning activity management system (LAMS). In G. Crisp, D. Thiele, I. Scholten, S. Barker, & J. Baron (Eds)., Proceedings of the 20th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE), 7–10 December 2003, Adelaide, Australia (pp. 593–596). Wollongong, N.S.W.: Australasian Society for Computers in Learning in Tertiary Education.
Dalziel, J. (2015). Learning design: Conceptualizing a framework for teaching and learning online. New York, NY: Routledge.
Ferguson, R. (2012). Learning analytics: Drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5-6), 304–317. https://dx.doi.org/10.1504/IJTEL.2012.051816
Gašević, D., Dawson, S., Rogers, T., & Gasevic, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education, 28, 68–84. https://dx.doi.org/10.1016/j.iheduc.2015.10.002
Goodyear, P. (2015). Teaching as design. HERDSA Review of Higher Education, 2, 27–50.
Griffiths, D. (2017). The use of models in learning design and learning analytics. Interaction Design and Architecture(s) Journal-IxD&A, 33, 113–133. Retrieved from http://www.mifav.uniroma2.it/inevent/events/idea2010/doc/33_6.pdf
Hernández-Leo, D., Asensio-Pérez, J. I., Derntl, M., Pozzi, F., Chacón, J., Prieto, L. P., & Persico, D. (2018). An integrated environment for learning design. Frontiers in ICT, 5(9). https://dx.doi.org/10.3389/fict.2018.00009
Ifenthaler, D., Gibson, D., & Dobozy, E. (2018). Informing learning design through analytics: Applying network graph analysis. Australasian Journal of Educational Technology, 34(2). https://dx.doi.org/10.14742/ajet.3767
Koper, R., & Manderveld, J. (2004). Educational modelling language: Modelling reusable, interoperable, rich and personalised units of learning. British Journal of Educational Technology, 35(5), 537–551. https://dx.doi.org/10.1111/j.0007-1013.2004.00412.x
Kreijns, K., Kirschner, P. A., & Jochems, W. (2003). Identifying the pitfalls for social interaction in computer-supported collaborative learning environments: A review of the research. Computers in Human Behavior, 19(3), 335–353. https://dx.doi.org/10.1016/S0747-5632(02)00057-2
Kruskal, J. B. (1964). Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika, 29(1), 1–27. https://dx.doi.org/10.1007/BF02289565
Laurillard, D., Kennedy, E., Charlton, P., Wild, J., & Dimakopoulos, D. (2018). Using technology to develop teachers as designers of TEL: Evaluating the learning designer. British Journal of Educational Technology, 49(6), 1044–1058. https://dx.doi.org/10.1111/bjet.12697
Law, N., Li, L., Herrera, L. F., Chan, A., & Pong, T.-C. (2017). A pattern language based learning design studio for an analytics informed inter-professional design community. Interaction Design and Architecture(s), 33, 92–112. Retrieved from http://www.mifav.uniroma2.it/inevent/events/idea2010/doc/33_5.pdf
Lockyer, L., & Dawson, S. (2011). Learning designs and learning analytics. Proceedings of the 1st International Conference on Learning Analytics and Knowledge (LAK ’11), 27 February–1 March 2011, Banff, AB, Canada (pp. 153–156). New York, NY: ACM. https://dx.doi.org/10.1145/2090116.2090140
Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an “early warning system” for educators: A proof of concept. Computers & Education, 54(2), 588–599. https://dx.doi.org/10.1016/j.compedu.2009.09.008
Mangaroska, K., & Giannakos, M. N. (2018). Learning analytics for learning design: A systematic literature review of analytics-driven design to enhance learning. IEEE Transactions on Learning Technologies, 12(4), 516–534. https://dx.doi.org/10.1109/TLT.2018.2868673
Mittelmeier, J., Rienties, B., Tempelaar, D., & Whitelock, D. (2018). Overcoming cross-cultural group work tensions: Mixed student perspectives on the role of social relationships. Higher Education, 75(1), 149–166. https://dx.doi.org/10.1007/s10734-017-0131-3
Mor, Y., Ferguson, R., & Wasson, B. (2015). Editorial: Learning design, teacher inquiry into student learning and learning analytics: A call for action. British Journal of Educational Technology, 46(2), 221–229. https://dx.doi.org/10.1111/bjet.12273
Newman, M. E. (2001). Scientific collaboration networks. II. Shortest paths, weighted networks, and centrality. Physical Review E, 64(1), 016132. https://dx.doi.org/10.1103/physreve.64.016132
Nguyen, Q., Huptych, M., & Rienties, B. (2018). Linking students’ timing of engagement to learning design and academic performance. Proceedings of the 8th International Conference on Learning Analytics & Knowledge (LAK ’18), 5–9 March 2018, Sydney, Australia (pp. 141–150). New York, NY: ACM. https://dx.doi.org/10.1145/3170358.3170398
Nguyen, Q., Rienties, B., & Toetenel, L. (2017). Unravelling the dynamics of instructional practice: A longitudinal study on learning design and VLE activities. Proceedings of the Seventh International Learning Analytics & Knowledge Conference (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 168–177). New York, NY: ACM. https://dx.doi.org/10.1145/3027385.3027409
Nguyen, Q., Rienties, B., Toetenel, L., Ferguson, R., & Whitelock, D. (2017). Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates. Computers in Human Behavior, 76, 703–714. https://dx.doi.org/10.1016/j.chb.2017.03.028
Nguyen, Q., Thorne, S., & Rienties, B. (2018). How do students engage with computer-based assessments: Impact of study breaks on intertemporal engagement and pass rates. Behaviormetrika, 45(2), 597–614. https://dx.doi.org/10.1007/s41237-018-0060-1
Persico, D., & Pozzi, F. (2015). Informing learning design with learning analytics to improve teacher inquiry. British Journal of Educational Technology, 46(2), 230–248. https://dx.doi.org/10.1111/bjet.12207
Persico, D., Pozzi, F., & Goodyear, P. (2018). Teachers as designers of TEL interventions. British Journal of Educational Technology, 49(6), 975–980. https://dx.doi.org/10.1111/bjet.12706
Rienties, B., & Toetenel, L. (2016). The impact of learning design on student behaviour, satisfaction and performance: A cross-institutional comparison across 151 modules. Computers in Human Behavior, 60, 333–341. https://dx.doi.org/10.1016/j.chb.2016.02.074
Rienties, B., Toetenel, L., & Bryan, A. (2015). Scaling up learning design: Impact of learning design activities on LMS behavior and performance. Proceedings of the Fifth International Conference on Learning Analytics and Knowledge (LAK ’15), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 315–319). New York, NY: ACM. https://dx.doi.org/10.1145/2723576.2723600
Thorpe, M. (2002). Rethinking learner support: The challenge of collaborative online learning. Open Learning: The Journal of Open, Distance and e-Learning, 17(2), 105–119. https://dx.doi.org/10.1080/02680510220146887a
Toetenel, L., & Rienties, B. (2016a). Analysing 157 learning designs using learning analytic approaches as a means to evaluate the impact of pedagogical decision making. British Journal of Educational Technology, 47(5), 981–992. https://dx.doi.org/10.1111/bjet.12423
Toetenel, L., & Rienties, B. (2016b). Learning design — Creative design to visualise learning activities. Open Learning: The Journal of Open, Distance and e-Learning, 31(3), 233–244. https://dx.doi.org/10.1080/02680513.2016.1213626
Ullmann, T. D. (2019). Automated analysis of reflection in writing: Validating machine learning approaches. International Journal of Artificial Intelligence in Education, 29(2), 217–257. https://dx.doi.org/10.1007/s40593-019-00174-2
Whitelock, D., Thorpe, M., & Galley, R. (2015). Student workload: A case study of its significance, evaluation and management at the Open University. Distance Education, 36(2), 161–176. https://dx.doi.org/10.1080/01587919.2015.1055059
Wise, A. F., Cui, Y., Jin, W., & Vytasek, J. (2017). Mining for gold: Identifying content-related MOOC discussion threads across domains through linguistic modeling. The Internet and Higher Education, 32, 11–28. https://dx.doi.org/10.1016/j.iheduc.2016.08.001
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2020 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
