Curriculum Modelling and Learner Simulation as a Tool in Curriculum (Re)Design
DOI:
https://doi.org/10.18608/jla.2022.7499Keywords:
computer-based modelling, simulation, curriculum design, research paperAbstract
Learning analytics (LA) provides tools to analyze historical data with the goal of better understanding how curricular structures and features have impacted student learning. Forward-looking curriculum design, however, frequently involves a degree of uncertainty. Historical data may be unavailable, a contemplated modification to curriculum may be unprecedented, or we may lack data regarding particular learner populations. To address this need, we propose using curriculum modelling and learner simulation (CMLS), which relies on well-established modelling theory and software to represent an existing or contemplated curriculum. The resulting model incorporates relevant research-based principles of learning to individually simulate learners and estimate their learning achievement as they move through the modelled curriculum. Results reflect both features of the curriculum (e.g., time allocated to different learning outcomes), learner profiles, and the natural variability of learners. We describe simulations with two versions of a college-level curriculum, explaining how results from simulations informed curriculum redesign work. We conclude with commentary on generalizing these methods, noting both theoretical and practical benefits of CMLS for curriculum (re)design.
References
ABET. (2018). Criteria for accrediting engineering programs. Accreditation Board for Engineering and Technology (ABET), Baltimore, MD. https://www.abet.org/accreditation/accreditation-criteria/criteria-for-accrediting-engineering-programs-2018-2019/#GC2
AIS Group. (2019). CPN Tools (Version 4.0.1). [Computer software]. Eindhoven University of Technology, Netherlands: AIS Group. http://cpntools.org/
Arthur, W. Jr., Bennett, W. Jr., Stanush, P. L., & McNelly, T. L. (1998). Factors that influence skill decay and retention: A quantitative review and analysis. Human Performance, 11(1), 57–101. https://doi.org/10.1207/s15327043hup1101_3
Ball, D. L., & Forzani, F. M. (2011). Building a common core for learning to teach. American Educator, 35(2), 17–21, 38–39.
Besterfield-Sacre, M., Shuman, L., Wolfe, H., Atman, C., Mcgourty, J., Miller, R., Olds, B., & Rogers, G. (2000). Defining the outcomes: A framework for EC-2000. IEEE Transactions on Education, 43(2), 100–110.
Brown, M., DeMonbrun, R. M., & Teasley, S. (2018). Taken together: Conceptualizing students’ concurrent course enrollment across the post-secondary curriculum using temporal analytics. Journal of Learning Analytics, 5(3), 60–72. https://doi.org/10.18608/jla.2018.53.5
Burin, D., Irrazabal, N., Ricle, I., Saux, G., & Barreyro, J. (2018). Self-reported internet skills, previous knowledge and working memory in text comprehension in e-learning. International Journal of Educational Technology in Higher Education, 15(1), 1–16. https://doi.org/10.1186/s41239-018-0099-9
Bursens, P., Donche, V., Gijbels, D., & Spooren, P. (2018). Simulations of decision-making as active learning tools: Design and effects of political science simulations. Springer International Publishing. https://doi.org/10.1007/978-3-319-74147-5
Carpenter, S. K., Cepeda, N. J., Rohrer, D., Kang, H. K., & Pashler, H. (2012). Using spacing to enhance diverse forms of learning: Review of recent research and implications for instruction. Educational Psychology Review, 24, 369–378. https://doi.org/10.1007/s10648-012-9205-z
Chou, C.-Y., Tseng, S.-F., Chih, W.-C., Chen, Z.-H., Chao, P.-Y., Lai, K. R., Chan, C.-L., Yu, L.-C., & Lin, Y.-L. (2017). Open student models of core competencies at the curriculum level: Using learning analytics for student reflection. IEEE Transactions on Emerging Topics in Computing, 5(1), 32–44. https://doi.org/10.1109/tetc.2015.2501805
Cronbach, L., & Snow, R. (1977). Aptitudes and instructional methods: A handbook for research on interactions. Irvington Publishers.
Ferguson, C. J. (2009). An effect size primer: A guide for clinicians and researchers. Professional Psychology: Research and Practice, 40(5), 532–538. https://doi.org/10.1037/14805-020
Follmer, D., Fang, S., Clariana, R., Meyer, B., & Li, P. (2018). What predicts adult readers’ understanding of STEM texts? Reading & Writing, 31(1), 185–214. https://doi.org/10.1007/s11145-017-9781-x
Forzani, F. M. (2014). Understanding “core practices” and “practice-based” teacher education: Learning from the past. Journal of Teacher Education, 65(4), 357–368. https://doi.org/10.1177/0022487114533800
Fredrick, W. C., & Walberg, H. J. (1980). Learning as a function of time. Journal of Education Research, 73(4), 183–194. https://doi.org/10.1080/00220671.1980.10885233
Fuchs, W. W., Fahsl, A. J., & James, S. M. (2014). Redesigning a special education teacher-preparation program: The rationale, process, and objectives. The New Educator, 10(2), 145–152. https://doi.org/10.1080/1547688X.2014.898493
Gijlers, H., & de Jong, T. (2013). Using concept maps to facilitate collaborative simulation-based inquiry learning. The Journal of the Learning Sciences, 22(3), 340–374. https://doi.org/10.1080/10508406.2012.748664
Gogate, L., & Hollich, G. (2013). Theoretical and computational models of word learning: Trends in psychology and artificial intelligence. IGI Global. https://doi.org/10.4018/978-1-4666-2973-8
Gonzalez-Brenes, J. P., & Huang, Y. (2015). Using data from real and simulated learners to evaluate adaptive tutoring systems. In J. Boticario & K. Muldner (Eds.), Proceedings of the Workshops at the 17th International Conference on Artificial Intelligence in Education (AIED 2015), 22–26 June 2015, Madrid, Spain (Vol. 5, pp. 31–34). http://ceur-ws.org/Vol-1432/sl_pap4.pdf
Gottipati, S., & Shankararaman, V. (2017). Competency analytics tool: Analyzing curriculum using course competencies. Education and Information Technologies, 23(1), 41–60. https://doi.org/10.1007/s10639-017-9584-3
Gressman, P. T., & Peck, J. R. (2020). Simulating COVID-19 in a university environment. Mathematical Biosciences, 328, 108436. https://doi.org/10.1016/j.mbs.2020.108436
Gromada, A., & Shewbridge, C. (2016). Student learning time: A literature review. OECD Education Working Papers, No. 127. OECD Publishing. https://doi.org/10.1787/5jm409kqqkjh-en
Guéraud, S., Walsh, E., Cook, A., & O’Brien, E. (2018). Validating information during reading: The effect of recency. Journal of Research in Reading, 41(S1), S85–S101. https://doi.org/10.1111/1467-9817.12244
Hale, J. A. (2008). A guide to curriculum mapping. Corwin Press.
Heileman, G. L., & Hickman, M., & Slim, A., & Abdallah, C. T. (2017, June). Characterizing the complexity of curricular patterns in engineering programs. Paper presented at the Annual Conference and Exposition of the American Society for Engineering Education (ASEE 2017), 24–28 June 2017, Columbus, OH, USA. https://doi.org/10.18260/1-2--28029
Hershkovitz, A., Knight, S., Jovanović, J., Dawson, S., & Gašević, D. (2017). Research with simulated data. Journal of Learning Analytics, 4(1), 1–2. https://doi.org/10.18608/jla.2017.41.1
Hilliger, I., Aguirre, C., Miranda, C., Celis, S., & Pérez-Sanagustín, M. (2020). Design of a curriculum analytics tool to support continuous improvement processes in higher education. Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK ’20), 23–27 March 2020, Frankfurt, Germany (pp. 181–186). ACM Press. https://doi.org/10.1145/3375462.3375489
Hsieh, S., & Hsu, L. (2013). An outcome-based evaluation of nursing competency of baccalaureate senior nursing students in Taiwan. Nurse Education Today, 33(12), 1536–1545. https://doi.org/10.1016/j.nedt.2013.01.012
Jacobs, H. H. (2004). Getting results with curriculum mapping. Association for Supervision and Curriculum Development.
Janes, J., Dunlosky, J., Rawson, K., & Jasnow, A. (2020). Successive relearning improves performance on a high‐stakes exam in a difficult biopsychology course. Applied Cognitive Psychology, 34(5), 1118–1132. https://doi.org/10.1002/acp.3699
Jensen, F. (2007). Introduction to computational chemistry (2nd ed.). John Wiley & Sons Ltd.
Jensen, K., & Kristensen, L. M. (2009). Coloured Petri nets. Springer-Verlag.
Kang, S. H. K. (2017). The benefits of interleaved practice for learning. In J. C. Horvath, J. M. Lodge & J. Hattie (Eds.), From the laboratory to the classroom: Translating science of learning for teachers (pp. 79–93). Routledge.
Kendeou, P., & van den Broek, P. (2007). The effects of prior knowledge and text structure on comprehension processes during reading of scientific texts. Memory & Cognition, 35(7), 1567–1577. https://doi.org/10.3758/BF03193491
Kostons, D., & van der Werf, G. (2015). The effects of activating prior topic and metacognitive knowledge on text comprehension scores. British Journal of Educational Psychology, 85(3), 264–275. https://doi.org/10.1111/bjep.12069
Kuang, X., Eysink, T. H., & de Jong, T. (2020). Effects of providing partial hypotheses as a support for simulation-based inquiry learning. Journal of Computer Assisted Learning, 36(4), 487–501. https://doi.org/10.1111/jcal.12415
Lattuca, L. R., & Stark, J. S. (2009). Shaping the college curriculum (2nd ed.). Wiley.
Leong, C., Tse, S., Loh, K., & Hau, K. (2008). Text comprehension in Chinese children: Relative contribution of verbal working memory, pseudoword reading, rapid automatized naming, and onset-rime phonological segmentation. Journal of Educational Psychology, 100(1), 135–149. https://doi.org/10.1037/0022-0663.100.1.135
Lomazova, I. A., Mitsyuk, A. A., & Sharipova, A. M. (2021). Modeling MOOC learnflow with Petri net extensions [Preprint manuscript]. https://arxiv.org/pdf/2111.04419.pdf
Mapelli, M., & Mayer, L. (2012). Ring galaxies from off-centre collisions. Monthly Notices of the Royal Astronomical Society, 420, 1158–1166. https://doi.org/10.1111/j.1365-2966.2011.20098.x
McEneaney, J. E. (2016). Simulation-based evaluation of learning sequences for instructional technologies. Instructional Science, 44(1), 87–106. https://doi.org/10.1007/s11251-016-9369-x
McEneaney, J. E. (2019). Modeling objectives-based learning in a mechanical engineering program [Unpublished manuscript]. Department of Reading and Language Arts, Oakland University, Rochester, MI.
Méndez, G., Ochoa, X., Chiluiza, K., & Wever, B. (2014). Curricular design analysis: A data‐driven perspective. Journal of Learning Analytics, 1(3), 84–119. https://doi.org/10.18608/jla.2014.13.6
Milner, R., Tofte, M., Harper, R., & MacQueen, D. (1997). The definition of standard ML (Revised edition). MIT Press. https://doi.org/10.7551/mitpress/2319.001.0001
Minsky, M. L., & Papert S. A. (1969). Perceptrons. MIT Press.
Munguia, P., & Brennan, A. (2020). Scaling the student journey from course-level information to program-level progression and graduation: A model. Journal of Learning Analytics, 7(2), 84–94. https://doi.org/10.18608/jla.2020.72.5
Morcke, A. M., Dornan, T., & Eika, B., (2013). Outcome (competency) based education: An exploration of its origins, theoretical basis, and empirical evidence. Advances in Health Sciences Education, 18, 851–863. https://doi.org/10.1007/s10459-012-9405-9
Otero, V. K. (2006). Moving beyond the “get it or don’t” conception of formative assessment. Journal of Teacher Education, 57(3), 247–255. https://doi.org/10.1177/0022487105285963
Pavlik, P. I., & Anderson, J. R. (2005). Practice and forgetting effects on vocabulary memory: An activation-based model of the spacing effect. Cognitive Science, 29, 559–586. https://doi.org/10.1207/s15516709cog0000_14
Pearson, P. D., Hansen, J., & Gordon, C. (1979). The effect of background knowledge on young children’s comprehension of explicit and implicit information. Journal of Reading Behavior, 11(3), 201–209. https://doi.org/10.1080/10862967909547324
Pelánek, R., Řihák, J., & Papoušek, J. (2016). Impact of data collection on interpretation and evaluation of student models. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼ16), 25–29 April 2016, Edinburgh, UK (pp. 40–47). ACM Press. https://doi.org/10.1145/2883851.2883868
Pilotti, M., Chodorow, M., & Petrov, R. (2009). The usefulness of retrieval practice and review-only practice for answering conceptually related test questions. The Journal of General Psychology, 136(2), 179–204. https://doi.org/10.3200/GENP.136.2.179-204
Salazar-Fernandez, J. P., Sepúlveda, M., Munoz-Gama, J., & Nussbaum, M. (2021). Curricular analytics to characterize educational trajectories in high-failure rate courses that lead to late dropout. Applied Sciences, 11(4), 1436. https://doi.org/10.3390/app11041436
Schneider, W., Körkel, J., & Weinert, F. E. (1989). Domain-specific knowledge and memory performance: A comparison of high- and low-aptitude children. Journal of Educational Psychology, 81(3), 306–312. https://doi.org/10.1037/0022-0663.81.3.306
Snow, R. (1989). Aptitude-treatment interaction as a framework for research on individual differences in learning. In P. Ackerman, R. J. Sternberg & R. Glaser (Eds.), Learning and individual differences (pp. 13–59). W. H. Freeman.
Spady, W. G. (1994). Outcome-based education: Critical issues and answers. American Association of School Administrators.
Spady, W. G., & Marshall, K. (1991). Beyond traditional outcome-based education. Educational Leadership, 49(2), 67–72.
Tan, K., Chong, M., Subramaniam, P., & Wong, L. (2018). The effectiveness of outcome-based education on the competencies of nursing students: A systematic review. Nurse Education Today, 64, 180–189. https://doi.org/10.1016/j.nedt.2017.12.030
TeachingWorks (2019). TeachingWorks, University of Michigan, School of Education, 2017–2018 Annual Report. Ann Arbor. https://report.teachingworks.org/assets/img/TeachingWorks%202017_2018%20Annual%20Report.pdf
Thalmann, D., & Musse, S. R. (2013). Crowd simulation (2nd ed.). Springer.
Tshai, K., Ho, J., Yap, E., & Ng, H. (2014). Outcome-based education: The assessment of programme educational objectives for an engineering undergraduate degree. Engineering Education, 9(1), 74–85. https://doi.org/10.11120/ened.2014.00020
Ullman, J. D. (1998). Elements of ML programming. Prentice Hall.
Weinstein, Y., Madan, C., & Sumeracki, M. (2018). Teaching the science of learning. Cognitive Research: Principles and Implications, 3(1), 1–17. https://doi.org/10.1186/s41235-017-0087-y
Wetherick, N. E. (1992). Cognition and simulation. Behavioral and Brain Sciences, 15(3) 462–463. https://doi.org/10.1017/S0140525X00069727
Winsberg, E. B. (2010). Science in the age of computer simulation. University of Chicago Press.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
TEST
