Network Analytics to Unveil Links of Learning Strategies, Time Management, and Academic Performance in a Flipped Classroom
DOI:
https://doi.org/10.18608/jla.2023.7843Keywords:
learning analytics, learning strategies, self-regulated learning, time management, research paperAbstract
Preparatory learning tasks are considered critical for student success in flipped classroom courses. However, less is known regarding which learning strategies students use and when they use those strategies in a flipped classroom course. In this study, we aimed to address this research gap. In particular, we investigated mutual connections between learning strategies and time management, and their combined effects on students’ performance in flipped classrooms. To this end, we harnessed a network analytic approach based on epistemic network analysis (ENA) to analyze student trace data collected in an undergraduate engineering course (N = 290) with a flipped classroom design. Our findings suggest that high-performing students effectively used their study time and enacted learning strategies mainly linked to formative and summative assessment tasks. The students in the low-performing group enacted less diverse learning strategies and typically focused on video watching. We discuss several implications for research and instructional practice.
References
Abeysekera, L., & Dawson, P. (2015). Motivation and cognitive load in the flipped classroom: Definition, rationale and a call for research. Higher Education Research & Development, 34(1), 1–14. https://doi.org/10.1080/07294360.2014.934336
Ahmad Uzir, N., Gasˇevic´, D., Jovanovic´, J., Matcha, W., Lim, L.-A., & Fudge, A. (2020). Analytics of time management and learning strategies for effective online learning in blended environments. In Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK 2020), 23–27 March 2020, Frankfurt, Germany (pp. 392–401). ACM. https://doi.org/https://doi.org/10.1145/3375462.3375493
Bergmann, J., & Sams, A. (2012). Flip your classroom. Reach every student in every classroom, every day. International Society for Technology in Education.
Bernacki, M. L., Chavez, M. M., & Uesbeck, P. M. (2020). Predicting achievement and providing support before STEM majors begin to fail. Computers and Education, 158, 103999. https://doi.org/10.1016/j.compedu.2020.103999
Bernard, R. M., Borokhovski, E., Schmid, R. F., Tamim, R. M., & Abrami, P. C. (2014). A meta-analysis of blended learning and technology use in higher education: From the general to the applied. Journal of Computing in Higher Education, 26(1), 87–122. https://doi.org/10.1007/s12528-013-9077-3
Bos, N., & Brand-Gruwel, S. (2016). Student differences in regulation strategies and their use of learning resources. In Proceedings of the Sixth International Conference on Learning Analytics and Knowledge (LAK 2016), 25–29 April 2016, Edinburgh, UK (pp. 344–353). ACM. https://doi.org/10.1145/2883851.2883890
Broadbent, J., & Poon, W. L. (2015). Self-regulated learning strategies and academic achievement in online higher education learning environments: A systematic review. Internet and Higher Education, 27, 1–13. https://doi.org/10.1016/j.iheduc.2015.04.007
Carolan, B. V. (2013). Social network analysis and education: Theory, methods & applications. Sage Publications. https://doi.org/10.4135/9781452270104
Carpenter, S. K., Pashler, H., Wixted, J. T., & Vul, E. (2008). The effects of tests on learning and forgetting. Memory & Cognition, 36(2), 438–448. https://doi.org/10.3758/MC.36.2.438
Cho, M. H., & Kim, B. J. (2013). Students’ self-regulation for interaction with others in online learning environments. Internet and Higher Education, 17(1), 69–75. https://doi.org/10.1016/j.iheduc.2012.11.001
Cho, M. H., & Shen, D. (2013). Self-regulation in online learning. Distance Education, 34(3), 290–301. https://doi.org/10.1080/01587919.2013.835770
Chun Chu, A. H., & Choi, J. N. (2005). Rethinking procrastination: Positive effects of “active” procrastination behavior on attitudes and performance. Journal of Social Psychology, 145(3), 245–264. https://doi.org/10.3200/SOCP.145.3.245-264
Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155. https://doi.org/10.1037/0033-2909.112.1.155
Dent, A. L., & Koenka, A. C. (2016). The relation between self-regulated learning and academic achievement across childhood and adolescence: A meta-analysis. Educational Psychology Review, 28(3), 425–474. https://doi.org/10.1007/s10648-015-9320-8
Douglas, H. E., Bore, M., & Munro, D. (2016). Coping with university education: The relationships of time management behaviour and work engagement with the five factor model aspects. Learning and Individual Differences, 45, 268–274. https://doi.org/10.1016/j.lindif.2015.12.004
Dunlosky, J. (2013). Strengthening the student toolbox. American Educator, 37(3), 12–21. https://www.aft.org/ae/fall2013/dunlosky
Fan, Y., Tan, Y., Rakovic´, M., Wang, Y., Cai, Z., Shaffer, D. W., & Gasˇevic´, D. (2022). Dissecting learning tactics in MOOC using ordered network analysis. Journal of Computer Assisted Learning, 39(1), 154–166. https://doi.org/10.1111/jcal.12735 Fincham, E., Gasˇevic´, D., Jovanovic´, J., & Pardo, A. (2018). From study tactics to learning strategies: An analytical method for extracting interpretable representations. IEEE Transactions on Learning Technologies, 12(1), 59–72. https://doi.org/10.1109/TLT.2018.2823317
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410–8415. https://doi.org/10.1073/pnas.1319030111
Gaševic´, D., Dawson, S., Rogers, T., & Gaševic´, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. Internet and Higher Education, 28, 68–84. https://doi.org/10.1016/j.iheduc.2015.10.002
Gaševic´, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1), 64–71. https://doi.org/10.1007/s11528-014-0822-x
Gaševic´, D., Jovanovic´, J., Pardo, A., & Dawson, S. (2017). Detecting learning strategies with analytics: Links with self-reported measures and academic performance. Journal of Learning Analytics, 4(2), 113–128. https://doi.org/10.18608/jla.2017.42.10
Gilboy, M. B., Heinerichs, S., & Pazzaglia, G. (2015). Enhancing student engagement using the flipped classroom. Journal of Nutrition Education and Behavior, 47(1), 109–114. https://doi.org/10.1016/j.jneb.2014.08.008
Goedhart, N. S., Blignaut-van Westrhenen, N., Moser, C., & Zweekhorst, M. B. (2019). The flipped classroom: Supporting a diverse group of students in their learning. Learning Environments Research, 22(2), 297–310. https://doi.org/10.1007/s10984-019-09281-2
Greene, J. A. (2020). Building upon synergies among self-regulated learning and formative assessment research and practice. Assessment in Education: Principles, Policy and Practice, 27(4), 463–476. https://doi.org/10.1080/0969594X.2020.1802225
Grosas, A. B., Raju, S. R., Schuett, B. S., Chuck, J.-A., & Millar, T. J. (2014). Determining if active learning through a formative assessment process translates to better performance in summative assessment. Studies in Higher Education, 5079(9), 1595–1611. https://doi.org/10.1080/03075079.2014.988704
Hadwin, A. F., Nesbit, J. C., Jamieson-Noel, D., Code, J., & Winne, P. H. (2007). Examining trace data to explore self-regulated learning. Metacognition and Learning, 2(2-3), 107–124. https://doi.org/10.1007/s11409-007-9016-7
He, W., Holton, A., Farkas, G., & Warschauer, M. (2016). The effects of flipped instruction on out-of-class study time, exam performance, and student perceptions. Learning and Instruction, 45, 61–71. https://doi.org/10.1016/j.learninstruc.2016.07.001
Heinerichs, S., Pazzaglia, G., & Gilboy, M. B. (2016). Using flipped classroom components in blended courses to maximize student learning. 11(1), 54–57. https://doi.org/10.4085/110154
Hintzman, D. L. (1974). Theoretical implications of the spacing effect. In R. L. Solso (Ed.), Theories in cognitive psychology: The Loyola Symposium. Lawrence Erlbaum.
Iqbal, S., Rakovic, M., Chen, G., Li, T., Ferreira Mello, R., Fan, Y., Fiorentino, G., Radi Aljohani, N., & Gaševic´, D. (2023). Towards automated analysis of rhetorical categories in students essay writings using Bloom’s taxonomy. In Proceedings of the 13th International Conference on Learning Analytics and Knowledge (LAK 2023), 13–17 March 2023, Arlington, Texas (pp. 418–429). ACM. https://doi.org/10.1145/3576050.3576112
Jovanovic, J., Gaševic´, D., Dawson, S., Pardo, A., & Mirriahi, N. (2017). Learning analytics to unveil learning strategies in a flipped classroom. Internet and Higher Education, 33, 74–85. https://doi.org/10.1016/j.iheduc.2017.02.001
Khan, I., & Pardo, A. (2006). Data2U: Scalable real time student feedback in active learning environments. In Proceedings of the Sixth International Conference on Learning Analytics and Knowledge (LAK 2016), 25–29 April 2016, Edinburgh, UK (pp. 249–253). https://doi.org/10.1145/2883851.2883911
Kim, S., Fernandez, S., & Terrier, L. (2017). Procrastination, personality traits, and academic performance: When active and passive procrastination tell a different story. Personality and Individual Differences, 108, 154–157. https://doi.org/10.1016/j.paid.2016.12.021
Kizilcec, R. F., Pérez-Sanagustín, M., & Maldonado, J. J. (2017). Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses. Computers and Education, 104, 18–33. https://doi.org/10.1016/j.compedu.2016.10.001
Klassen, R. M., Krawchuk, L. L., & Rajani, S. (2008). Academic procrastination of undergraduates: Low self-efficacy to self-regulate predicts higher levels of procrastination. Contemporary Educational Psychology, 33(4), 915–931. https://doi.org/10.1016/j.cedpsych.2007.07.001
Lee, Y., & Choi, J. (2011). A review of online course dropout research: Implications for practice and future research. Educational Technology Research and Development, 59(5), 593–618. https://doi.org/10.1007/s11423-010-9177-y
Levy, Y., & Ramim, M. M. (2012). A study of online exams procrastination using data analytics techniques. Interdisciplinary Journal of E-Learning & Learning Objects, 8, 97–113. https://www.ijello.org/Volume8/IJELLOv8p097-113Levy0804.pdf
Li, T., Fan, Y., Tan, Y., Wang, Y., Singh, S., Li, X., Rakovic´, M., Van Der Graaf, J., Lim, L., Yang, B., Molenaar, I., Bannert, M., Moore, J., Swiecki, Z., Tsai, Y.-S., Shaffer, D. W., & Gasˇevic´, D. (2023). Analytics of self-regulated learning scaffolding: Effects on learning processes. Frontiers in Psychology, 14. https://doi.org/10.3389/fpsyg.2023.1206696
Lin, C.-J., & Hwang, G.-J. (2018). A learning analytics approach to investigating factors affecting EFL students’ oral performance in a flipped classroom. Journal of Educational Technology & Society, 21(2), 205–219. https://drive.google.com/file/d/1BV8BdjlFsVZL1KndV4TEh6pyExfeuW3/view
Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing pedagogical action: Aligning learning analytics with learning design. American Behavioral Scientist, 57(10), 1439–1459. https://doi.org/10.1177/0002764213479367
Logan, J. M., Castel, A. D., Haber, S., & Viehman, E. J. (2012). Metacognition and the spacing effect: The role of repetition, feedback, and instruction on judgments of learning for massed and spaced rehearsal. Metacognition and Learning, 7(3), 175–195. https://doi.org/10.1007/s11409-012-9090-3
Luo, H., Millet, A., Alley, R., & Zuo, M. (2016). Dealing with ethical issues in MOOC design and delivery: A case study Blended Learning: Aligning Theory with Practices, 9757, 128–138. https://doi.org/10.1007/978-3-319-41165-1_12
Matcha, W., Gaševic´, D., Ahmad Uzir, N., Jovanovic´, J., & Pardo, A. (2019). Analytics of learning strategies: Associations with academic performance and feedback. In Proceedings of the Ninth International Conference on Learning Analytics and Knowledge (LAK 2019), 4–8 March 2019, Tempe, AZ (pp. 461–470). ACM. https://doi.org/10.1145/3303772.3303787
McLean, S., Attardi, S. M., Faden, L., & Goldszmidt, M. (2016). Flipped classrooms and student learning: Not just surface gains. Advances in Physiology Education, 40(1), 47–55. https://doi.org/10.1152/advan.00098.2015
Michinov, N., Brunot, S., Le Bohec, O., Juhel, J., & Delaval, M. (2011). Procrastination, participation, and performance in online learning environments. Computers and Education, 56(1), 243–252. https://doi.org/10.1016/j.compedu.2010.07.025
Nash, P., & Shaffer, D. W. (2012). Epistemic trajectories: Mentoring in a game design practicum. Instructional Science, 41(4), 745–771. https://doi.org/10.1007/s11251-012-9255-0
O’Flaherty, J., & Phillips, C. (2015). The use of flipped classrooms in higher education: A scoping review. Internet and Higher Education, 25, 85–95. https://doi.org/10.1016/j.iheduc.2015.02.002
Pardo, A., Han, F., & Ellis, R. A. (2017). Combining university student self-regulated learning indicators and engagement with online learning events to predict academic performance. IEEE Transactions on Learning Technologies, 10(1), 82–92. https://doi.org/10.1109/tlt.2016.2639508
Pardo, A., & Mirriahi, N. (2017). Design, deployment and evaluation of a flipped learning first year engineering course. In C. Reidsema, L. Kavanagh, R. Hadgraft, & N. Smith (Eds.), The flipped classroom: Practice and practices in higher education (pp. 177–191). Springer. https://doi.org/10.1007/978-981-10-3413-8 11
Pierce, R., & Fox, J. (2012). Vodcasts and active-learning exercises in a “flipped classroom” model of a renal pharmacotherapy module. American Journal of Pharmaceutical Education, 76(10), 196. https://doi.org/10.5688/ajpe7610196
Pintrich, P. R. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 451–502). https://doi.org/10.1016/B978-012109890-2/50043-3
Rakovic´, M., Bernacki, M. L., Greene, J. A., Plumley, R. D., Hogan, K. A., Gates, K. M., & Panter, A. T. (2022). Examining the critical role of evaluation and adaptation in self-regulated learning. Contemporary Educational Psychology, 68, 102027. https://doi.org/10.1016/j.cedpsych.2021.102027
Roediger III, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249–255. https://doi.org/10.1111/j.1467-9280.2006.01693.x
Roehl, A., Reddy, S. l., & Shannon, G. J. (2013). The flipped classroom: An opportunity to engage the millennial students through active learning strategies. Journal of Family and Consumer Sciences, 105(2), 44–49. https://www.ingentaconnect.com/contentone/aafcs/jfcs/2013/00000105/00000002/art00012
Rupp, A. A., Gushta, M., Mislevy, R. J., & Shaffer, D. W. (2010). Evidence-centered design of epistemic games: Measurement principles for complex learning environments. The Journal of Technology, Learning and Assessment, 8(4). https://ejournals.bc.edu/index.php/jtla/article/view/1623/1467
Rupp, A. A., Sweet, S. J., & Choi, Y. (2010). Modeling learning trajectories with epistemic network analysis: A simulation- based investigation of a novel analytic method for epistemic games. In R. Baker, A. Merceron, & P. I. Pavlik Jr. (Eds.), Proceedings of the Third International Conference on Educational Data Mining (EDM 2010), 11–13 June 2010, Pittsburgh, Pennsylvania (pp. 319–320). https://educationaldatamining.org/EDM2010/uploads/proc/2010%20Proceedings%20Preface,%20TOC.pdf
Sajid, M. R., Laheji, A. F., Abothenain, F., Salam, Y., AlJayar, D., & Obeidat, A. (2016). Can blended learning and the flipped classroom improve student learning and satisfaction in Saudi Arabia? International Journal of Medical Education, 7, 281–285. https://doi.org/10.5116/ijme.57a7.83d4
Saqr, M., Lo´pez-Pernas, S., Jovanovic´, J., & Gaševic´, D. (2023). Intense, turbulent, or wallowing in the mire: A longitudinal study of cross-course online tactics, strategies, and trajectories. The Internet and Higher Education, 57, 100902. https://doi.org/10.1016/j.iheduc.2022.100902
Schunk, D. H. (2012). Learning theories: An educational perspective (6th edition). Pearson.
Shaffer, D. W., Collier, W., & Ruis, A. R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3(3), 9–45. https://doi.org/10. 18608/jla.2016.33.3
Shaffer, D. W., & Gee, J. P. (2012). The right kind of GATE: Computer games and the future of assessment. In M. C. Mayrath, J. Clarke-Midura, D. H. Robinson, & G. Schraw (Eds.), Technology-based assessments for 21st century skills: Theoretical and practical implications from modern research (pp. 211–228). Information Age Publishing.
Shaffer, D. W., Hatfield, D., Svarovsky, G. N., Nash, P., Nulty, A., Bagley, E., Frank, K., Rupp, A. A., & Mislevy, R. (2009). Epistemic network analysis: A prototype for 21st-century assessment of learning. International Journal of Learning and Media, 1(2), 33–53. https://doi.org/10.1162/ijlm.2009.0013
Siadaty, M. (2016). Trace-based microanalytic measurement of self-regulated learning processes. Journal of Learning Analytics, 3(1), 183–214. https://doi.org/10.18608/jla.2016.31.11
Siadaty, M., Gaševic´, D., & Hatala, M. (2016). Associations between technological scaffolding and micro-level processes of self-regulated learning: A workplace study. Computers in Human Behavior, 55(Part B), 1007–1019. https://doi.org/10.1016/j.chb.2015.10.035
Soderstrom, N. C., & Bjork, R. A. (2015). Learning versus performance: An integrative review. Perspectives on Psychological Science, 10(2), 176–199. https://doi.org/10.1177/1745691615569000
Srivastava, N., Fan, Y., Rakovic, M., Singh, S., Jovanovic, J., van der Graaf, J., Lim, L., Surendrannair, S., Kilgour, J., Molenaar, I., Bannert, M., Moore, J., & Gaševic´, D. (2022). Effects of internal and external conditions on strategies of self-regulated learning: A learning analytics study. In Proceedings of the 12th International Conference on Learning Analytics and Knowledge (LAK 2022), 21–25 March 2022, online (pp. 392–403). ACM. https://doi.org/10.1145/3506860.3506972
Steel, P. (2007). The nature of procrastination: a meta-analytic and theoretical review of quintessential self-regulatory failure. Psychological Bulletin, 133(1), 65–94. https://doi.org/10.1037/0033-2909.133.1.65
Sun, Z., Xie, K., & Anderman, L. H. (2018). The role of self-regulated learning in students’ success in flipped undergraduate math courses. The Internet and Higher Education, 36, 41–53. https://doi.org/10.1016/j.iheduc.2017.09.003
Thai, T. N. T., Wever, B. D., & Valcke, M. (2017). The impact of a flipped classroom design on learning performance in higher education: Looking for the best “blend” of lectures and guiding questions with feedback. Computers & Education, 107, 1–27. https://doi.org/10.1016/j.compedu.2017.01.003
Tucker, B. (2012). The flipped classroom: Online instruction at home frees class time for learning. Education Next, 12(1), 82–83. https://www.educationnext.org/the-flipped-classroom/
Weinstein, C., & Mayer, R. (1983). The teaching of learning strategies. Innovation Abstracts, 5(32), 2–4. http://files.eric.ed.gov/fulltext/ED237180.pdf
Winne, P. H. (2005). A perspective on state-of-the-art research on self-regulated learning. Instructional Science, 33(5), 559–565. https://doi.org/10.1007/s11251-005-1280-9
Winne, P. H. (2013). Learning strategies, study skills, and self-regulated learning in postsecondary education. In M. B. Paulsen (Ed.), Higher education: Handbook of theory and research (pp. 377–403). Springer. https://doi.org/10.1007/978-94-007-5836-0_8
Winne, P. H. (2018). Theorizing and researching levels of processing in self-regulated learning. British Journal of Educational Psychology, 88(1), 9–20. https://doi.org/10.1111/bjep.12173
Winne, P. H. (2022). Modeling self-regulated learning as learners doing learning science: How trace data and learning analytics help develop skills for self-regulated learning. Metacognition and Learning, 17, 1–19. https://doi.org/10.1007/s11409-022-09305-y
Winne, P. H., & Azevedo, R. (2014). Metacognition. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (2nd edition, pp. 63–87). Cambridge University Press. https://doi.org/10.1017/CBO9781139519526.006
Winne, P. H., Gupta, L., & Nesbit, J. C. (1994). Exploring individual differences in studying strategies using graph theoretic statistics. Alberta Journal of Educational Research, 40(2), 177–93.
Winne, P. H., & Hadwin, A. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 277–304). Lawrence Erlbaum. https://psycnet.apa.org/record/1998-07283-011
Wolters, C. A. (2003). Regulation of motivation: Evaluating an underemphasized aspect of self-regulated learning. Educational Psychologist, 38(4), 189–205. https://doi.org/10.1207/S15326985EP3804_1
Yamada, M., Shimada, A., Okubo, F., Oi, M., Kojima, K., & Ogata, H. (2017). Learning analytics of the relationships among self-regulated learning, learning behaviors, and learning performance. Research and Practice in Technology Enhanced Learning, 12(1), 13. https://doi.org/10.1186/s41039-017-0053-9
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
