Working Together in Learning Analytics Towards the Co-Creation of Value
DOI:
https://doi.org/10.18608/jla.2019.62.2Keywords:
learning analytics; co-creation; co-design; participatory designAbstract
The value of technology lies not only with the service or functionality of the tool, but also with its subsequent value to the people who use it. New learning analytics (LA) software and platforms for capturing data and improving student learning are frequently introduced; however, they suffer from issues of adoption and continued usage by stakeholders. Scholars have previously suggested that it is not enough to introduce stakeholders (e.g., teachers and students) to LA technologies; they must also be a part of the LA creation and design process. In this paper, we will continue the ongoing work to clarify and compare different approaches of human-centred design through an overview of participatory frameworks in LA (co-design, co-creation). We will also present a case study using an LA tool used and developed within several Australian universities that was utilized over six years as an example of how LA designers can co-create dynamic platforms with teachers. Implications of participatory design frameworks in LA will also be presented through a discussion of the costs, challenges, and benefits of adopting human-centred design.
References
Abras, C., Maloney-Krichmar, D., & Preece, J. (2004). User-centered design. In W. Bainbridge (Ed.), Encyclopedia of human–computer interaction (pp. 445–456). Thousand Oaks, CA: Sage.
Arthars, N., Dollinger, M., Vigentini, L., Liu, D., Kondo, E., & King, D. (2019). Empowering teachers to personalize learning support. In D. Ifenthaler, D. K. Mah, & J. Y. K. Yau (Eds.), Utilizing learning analytics to support study success (pp. 223–248). Springer. https://dx.doi.org/10.1007/978-3-319-64792-0_13
Bergvall-Kåreborn, B., & Ståhlbrost, A. (2008). Participatory design: One step back or two steps forward? In Proceedings of the 10th Anniversary Conference on Participatory Design 2008 (PDC ’08) 1–4 October 2008, Bloomington, IN, USA (pp. 102–111). New York: AMC.
Blumenstein, M., Liu, D. Y. T., Richards, D., Leichtweis, S., & Stephens, J. M. (2018). Data-informed nudges for student engagement and success. In J. M. Lodge, J. C. Horvath, & L. Corrin (Eds.), Learning analytics in the classroom: Translating learning analytics research for teachers: Abingdon, UK: Routledge.
Brandt, E. (2006). Designing exploratory design games: A framework for participation in participatory design? In Proceedings of the 9th Conference on Participatory Design: Expanding Boundaries in Design (PDC ’06), 1–5 August 2006, Trento, Italy (Vol. 1, pp. 57–66). New York: AMC. https://dx.doi.org/10.1145/1147261.1147271
Bratteteig, T., & Wagner, I. (2012). Disentangling power and decision-making in participatory design. In Proceedings of the 12th Participatory Design Conference: Research Papers (PDC ’12), 12–16 August 2012, Roskilde, Denmark (Vol. 1, pp. 41–50). New York: AMC. https://dx.doi.org/10.1145/2347635.2347642
Buchanan, R. (2001). Human dignity and human rights: Thoughts on the principles of human-centered design. Design Issues, 17(3), 35–39. https://dx.doi.org/10.1162/074793601750357178
Buckingham Shum, S., & Ferguson, R. (2012). Social learning analytics. Journal of Educational Technology & Society, 15(3). https://dx.doi.org/10.1145/2330601.2330616
Buckingham Shum, S., & McKay, T. (2018). Architecting for learning analytics: Innovating for sustainable impact. EDUCAUSE REVIEW, March/April 2018, pp. 25–37. Retrieved at https://er.educause.edu/articles/2018/3/architecting-for-learning-analytics-innovating-for-sustainable-impact
Carroll, J. M. (1996). Encountering others: Reciprocal openings in participatory design and user-centered design. Human–Computer Interaction, 11(3), 285–290. https://dx.doi.org/10.1207/s15327051hci1103_5
Clement, A., & Van den Besselaar, P. (1993). A retrospective look at PD projects. Communications of the ACM, 36(6), 29–37. https://dx.doi.org/10.1145/153571.163264
Chai, C. S., Koh, E., Lim, C. P., & Tsai, C. C. (2014). Deepening ICT integration through multilevel design of technological pedagogical content knowledge. Journal of Computers in Education, 1(1), 1–17. https://dx.doi.org/10.1007/s40692-014-0002-1
Conole, G., & Culver, J. (2010). The design of Cloudworks: Applying social networking practice to foster the exchange of learning and teaching ideas and designs. Computers & Education, 54(3), 679–692. https://dx.doi.org/10.1016/j.compedu.2009.09.013
Cook-Sather, A. (2017). How pedagogical partnerships can build a sense of belonging, create a trusting classroom community, and spark hope. Teaching and Learning Together in Higher Education, 1(22), 1. http://repository.brynmawr.edu/tlthe/vol1/iss22/1
Corrin, L., & de Barba, P. (2015). How do students interpret feedback delivered via dashboards? In Proceedings of the 5th International Conference on Learning Analytics and Knowledge (LAK ʼ15), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 430–431). New York: ACM. https://dx.doi.org/10.1145/2723576.2723662
Dimitriadis, Y., & Goodyear, P. (2013). Forward-oriented design for learning: Illustrating the approach. Research in Learning Technology, 21(1). https://dx.doi.org/10.3402/rlt.v21i0.20290
Dollinger, M. (2018). Technology for the scalability of co-creation with students. In M. Campbell, J. Willems, C. Adachi, D. Blake, I. Doherty, S. Krishnan, S. Macfarlane, L. Ngo, M. O’Donnell, S. Palmer, L. Riddell, I. Story, H. Suri, ... J. Tai (Eds.), Open Oceans: Learning Without Borders. Proceedings of the 35th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE 2018), 25–28 November 2018, Geelong, Victoria, Australia (pp. 346–350).
Dollinger, M., & Lodge, J. M. (2018). Co-creation strategies for learning analytics. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK ’18), 5–9 March 2018, Sydney, NSW, Australia (pp. 97–101). New York: ACM. https://dx.doi.org/10.1145/3170358.3170372
Dollinger, M., Lodge, J. M., & Coates, H. (2018). Co-creation in higher education: Towards a conceptual model. Journal of Marketing for Higher Education, 28(2), 210–231. https://dx.doi.org/10.1080/08841241.2018.1466756
Drachsler, H., & Greller, W. (2012). The pulse of learning analytics understandings and expectations from the stakeholders. In S. Buckingham Shum, D. Gašević, & R. Ferguson (Eds.), Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK ʼLA), 29 April–2 May 2012, Vancouver, BC, Canada (pp. 120–129). New York: ACM. https://dx.doi.org/10.1145/2330601.2330634
Dyckhoff, A. L., Zielke, D., Bültmann, M., Chatti, M. A., & Schroeder, U. (2012). Design and implementation of a learning analytics toolkit for teachers. Journal of Educational Technology & Society, 15(3), 58–76.
Elias, T. (2011). Learning analytics: Definitions, processes and potential. Retrieved from http://learninganalytics.net/LearningAnalyticsDefinitionsProcessesPotential.pdf
Ferguson, R., Clow, D., Macfadyen, L., Essa, A., Dawson, S., & Alexander, S. (2014). Setting learning analytics in context: Overcoming the barriers to large-scale adoption. In Proceedings of the 4th International Conference on Learning Analytics and Knowledge (LAK ʼ14), 24–28 March 2014, Indianapolis, IN, USA (pp. 251–253). New York: ACM. https://dx.doi.org/10.1145/2567574.2567592
Fischer, G., Nakakoji, K., & Ye, Y. (2009). Metadesign: Guidelines for supporting domain experts in software development. IEEE software, 26(5). https://dx.doi.org/10.1109/ms.2009.134
Friesen, N. (2013). Learning analytics: Readiness and rewards. Canadian Journal of Learning and Technology, 39(4), 1–12. https://dx.doi.org/10.21432/t2j01b
Frow, P., Nenonen, S., Payne, A., & Storbacka, K. (2015). Managing co‐creation design: A strategic approach to innovation. British Journal of Management, 26(3), 463–483. https://dx.doi.org/10.1111/1467-8551.12087
Giacomin, J. (2014). What is human centred design? The Design Journal, 17(4), 606–623. https://dx.doi.org/10.2752/175630614x14056185480186
Gibson, D., & de Freitas, S. (2016). Exploratory analysis in learning analytics. Technology, Knowledge and Learning, 21(1), 5–19. https://dx.doi.org/10.1007/s10758-015-9249-5
Glucker, A. N., Driessen, P. P., Kolhoff, A., & Runhaar, H. A. (2013). Public participation in environmental impact assessment: Why, who and how? Environmental Impact Assessment Review, 43, 104–111. https://doi.org/10.1016/j.eiar.2013.06.003
Heath, J. (2014). Contemporary privacy theory contributions to learning analytics. Journal of Learning Analytics, 1(1), 140–149. https://dx.doi.org/10.18608/jla.2014.11.8
Hickey, D. T., Kelley, T. A., & Shen, X. (2014). Small to big before massive: Scaling up participatory learning analytics. In Proceedings of the 4th International Conference on Learning Analytics and Knowledge (LAK ʼ14), 24–28 March 2014, Indianapolis, IN, USA (pp. 93–97). New York: ACM. https://dx.doi.org/10.1145/2567574.2567626
Holstein, K., Hong, G., Tegene, M., McLaren, B. M., & Aleven, V. (2018). The classroom as a dashboard: Co-designing wearable cognitive augmentation for K–12 teachers. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK ’18), 5–9 March 2018, Sydney, NSW, Australia (pp. 79–88). New York: ACM. https://dx.doi.org/10.1145/3170358.3170377
Holstein, K., McLaren, B. M., & Aleven, V. (2017). Intelligent tutors as teachers’ aides: Exploring teacher needs for real-time analytics in blended classrooms. In Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 257–266). New York: ACM. https://dx.doi.org/10.1145/3027385.3027451
Hussain, S., Sanders, E. B. N., & Steinert, M. (2012). Participatory design with marginalized people in developing countries: Challenges and opportunities experienced in a field study in Cambodia. International Journal of Design, 6(2). http://www.ijdesign.org/index.php/IJDesign/article/view/1054/455
Jabri, M. (2017). Managing organizational change: Process, social construction and dialogue. Basingstoke, UK: Palgrave Macmillan.
Kensing, F., & Blomberg, J. (1998). Participatory design: Issues and concerns. Computer Supported Cooperative Work, 7(3–4), 167–185. https://dx.doi.org/10.1023/a:1008689307411
Khalil, M., & Ebner, M. (2016). De-identification in learning analytics. Journal of Learning Analytics, 3(1), 129–138. https://dx.doi.org/10.18608/jla.2016.31.8
Knight, D. B., Brozina, C., & Novoselich, B. (2016). An investigation of first-year engineering student and instructor perspectives of learning analytics approaches. Journal of Learning Analytics, 3(3), 215–238. https://dx.doi.org/10.18608/jla.2016.33.11
Knight, D. B., Brozina, C., Stauffer, E. M., Frisina, C., & Abel, T. (2015). Developing a learning analytics dashboard for undergraduate engineering using participatory design. In Proceedings of the 122nd Annual Conference of the American Society for Engineering Education (ASEE 2015), 13 June 2015, Seattle, WA, USA (p. 23824). https://dx.doi.org/10.18260/p.23824
Krippendorff, K. (2004). Intrinsic motivation and human-centred design. Theoretical Issues in Ergonomics Science, 5(1), 43–72. https://dx.doi.org/10.1080/1463922031000086717
Kujala, S. (2003). User involvement: A review of the benefits and challenges. Behaviour & Information Technology, 22(1), 1–16. https://dx.doi.org/10.1080/01449290301782
Liu, D. Y. T., Froissard, J. C., Richards, D., & Atif, A. (2015). An enhanced learning analytics plugin for Moodle: Student engagement and personalised intervention. In T. Reiners, B. R. von Konsky, D. Gibson, V. Chang, L. Irving, & K. Clarke (Eds.), Globally Connected, Digitally Enabled: Proceedings of the 32nd Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE 2015), 29 November–2 December 2015, Perth, Western Australia (pp. FP:168–FP:177).
Liu, D. Y. T., Taylor, C. E., Bridgeman, A. J., Bartimote-Aufflick, K., & Pardo, A. (2016). Empowering instructors through customizable collection and analyses of actionable information. Paper presented at the 1st Learning Analytics for Curriculum and Program Quality Improvement Workshop (PCLA 2016), 25 April 2016, Edinburgh, UK.
Liu, D. Y. T., Bartimote-Aufflick, K., Pardo, A., & Bridgeman, A. J. (2017). Data-driven personalization of student learning support in higher education. In A. Peña-Ayala (Ed.), Learning analytics: Fundaments, applications, and trends (pp. 143–169): Springer. https://dx.doi.org/10.1007/978-3-319-52977-6_5
Lodge, J. M., Alhadad, S. S., Lewis, M. J., & Gašević, D. (2017). Inferring learning from big data: The importance of a transdisciplinary and multidimensional approach. Technology, Knowledge and Learning, 22(3), 385–400. https://dx.doi.org/10.1007/s10758-017-9330-3
Macfadyen, L. P., & Dawson, S. (2012). Numbers are not enough: Why e‐learning analytics failed to inform an institutional strategic plan. Educational Technology & Society, 15(3), 149–163.
Maguire, M. (2001). Methods to support human-centred design. International Journal of Human–Computer Studies, 55(4), 587–634. https://dx.doi.org/10.1006/ijhc.2001.0503
Marjanovic, O. (2014). Using practitioner stories to design learning experiences in visual analytics. In Proceedings of the 2014 International SIGED IAIM Conference, 14 December 2014, Auckland, New Zealand. Association for Information Systems (AIS). https://aisel.aisnet.org/siged2014/4/
Martinez-Maldonado, R., Schneider, B., Charleer, S., Buckingham Shum, S., Klerkx, J., & Duval, E. (2016). Interactive surfaces and learning analytics: Data, orchestration aspects, pedagogical uses and challenges. In Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼ16), 25–29 April 2016, Edinburgh, UK (pp. 124–133). New York: ACM. https://dx.doi.org/10.1145/2883851.2883873
Norman, D. A., & Draper, S. W. (1986). User centered system design: New perspectives on human–computer interaction. Boca Raton, FL: CRC Press.
Open University (2014). Policy on Ethical use of Student Data for Learning Analytics. The Open University.
Payne, A. F., Storbacka, K., & Frow, P. (2008). Managing the co-creation of value. Journal of the Academy of Marketing Science, 36(1), 83–96. https://dx.doi.org/10.1007/s11747-007-0070-0
Prinsloo, P., & Slade, S. (2015). Student privacy self-management: Implications for learning analytics. In Proceedings of the 5th International Conference on Learning Analytics and Knowledge (LAK ʼ15), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 83–92). New York: ACM. https://dx.doi.org/10.1145/2723576.2723585
Prinsloo, P., & Slade, S. (2017). Ethics and learning analytics: Charting the (un)charted. In C. Lang, G. Siemens, A Wise, & D. Gašević (Eds.), Handbook of learning analytics (pp. 49–57). Beaumont, AB: Society for Learning Analytics Research. https://dx.doi.org/10.18608/hla17.004
Ramaswamy, V., & Ozcan, K. (2014). The co-creation paradigm. Palo Alto, CA: Stanford University Press.
Ranjan, K. R., & Read, S. (2016). Value co-creation: Concept and measurement. Journal of the Academy of Marketing Science, 44(3), 290–315. https://dx.doi.org/10.1007/s11747-014-0397-2
Rienties, B., & Toetenel, L. (2016). The impact of learning design on student behaviour, satisfaction and performance: A cross-institutional comparison across 151 modules. Computers in Human Behavior, 60, 333–341. https://dx.doi.org/10.1016/j.chb.2016.02.074
Robertson, T., & Simonsen, J. (2012). Challenges and opportunities in contemporary participatory design. Design Issues, 28(3), 3–9. https://dx.doi.org/10.1162/desi_a_00157
Rogers, E. M. (2003). The diffusion of innovations. 5th ed. New York: Free Press.
Sanders, E. B. N., Brandt, E., & Binder, T. (2010). A framework for organizing the tools and techniques of participatory design. In Proceedings of the 11th Biennial Participatory Design Conference (PDC ’10) 29 November–3 December 2010, Sydney, Australia (pp. 195–198). New York: AMC. https://dx.doi.org/10.1145/1900441.1900476
Sanders, E. B. N., & Stappers, P. J. (2008). Co-creation and the new landscapes of design. Co-design, 4(1), 5–18. https://dx.doi.org/10.1080/15710880701875068
Sangiorgi, D. (2012). Transformative services and transformation design. International Journal of Design, 5, 29–40. http://hdl.handle.net/11311/968237
Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York: Basic Books.
Schuler, D., & Namioka, A. (Eds.). (1993). Participatory design: Principles and practices. Boca Raton, FL: CRC Press.
Sclater, N. (2016). Developing a code of practice for learning analytics. Journal of Learning Analytics, 3(1), 16–42. https://dx.doi.org/10.18608/jla.2016.31.3
Siemens, G. (2013). Learning analytics: The emergence of a discipline. American Behavioral Scientist, 57(10), 1380–1400. https://dx.doi.org/10.1177/0002764213498851
Siemens, G., & Baker, R. S. J. d. (2012). Learning analytics and educational data mining: Towards communication and collaboration. In S. Buckingham Shum, D. Gašević, & R. Ferguson (Eds.), Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK ʼ12), 29 April–2 May 2012, Vancouver, BC, Canada (pp. 252–254). New York: ACM. https://dx.doi.org/10.1145/2330601.2330661
Thompson, K. (2013). Using micro-patterns of speech to predict the correctness of answers to mathematics problems: An exercise in multimodal learning analytics. In Proceedings of the 15th ACM International Conference on Multimodal Interaction (ICMI ’13), 9–13 December 2013, Sydney, Australia (pp. 591–598). New York: ACM. https://dx.doi.org/10.1145/2522848.2533792
Tsai, Y. S., & Gašević, D. (2017). Learning analytics in higher education — challenges and policies: A review of eight learning analytics policies. In Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 233–242). New York: ACM. https://dx.doi.org/10.1145/3027385.3027400
Vatrapu, R., Reimann, P., Hussain, A., & und Beratung, M. P. F. (2012). Towards teaching analytics: Repertory grids for formative assessment. In J. van Aalst, B. J. Reiser, C. Hmelo-Silver, & K. Thompson (Eds.), The Future of Learning: Proceedings of the 10th International Conference of the Learning Sciences (ICLS ʼ12), 2–6 July 2012, Sydney, Australia (Volume 2, Short Papers, Symposia, and Abstracts). International Society of the Learning Sciences.
Verbert, K., Govaerts, S., Duval, E., Santos, J. L., Van Assche, F., Parra, G., & Klerkx, J. (2014). Learning dashboards: An overview and future research opportunities. Personal and Ubiquitous Computing, 18(6), 1499–1514. https://dx.doi.org/10.1007/s00779-013-0751-2
Vigentini, L., Kondo, E., Samnick, K., Liu, D., King, D., & Bridgeman, A. (2017). Recipes for institutional adoption of a teacher-driven learning analytics tool: Case studies from three Australian universities. In H. Partridge, K. Davis, & J. Thomas (Eds.), Me, Us, IT! Proceedings of the 34th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE 2017), 4–6 December 2017, Toowoomba, Queensland, Australia (pp. 422–432).
Von Hippel, E. (2005). Democratizing innovation. Cambridge, MA: MIT press.
West, D., Heath, D., & Huijser, H. (2016). Let’s talk learning analytics: A framework for implementation in relation to student retention. Online Learning, 20(2), 30–50. https://dx.doi.org/10.24059/olj.v20i2.792
Yip, J., Ahn, J., Clegg, T., Bonsignore, E., Pauw, D., & Gubbels, M. (2014). It helped me do my science: A case of designing social media technologies for children in science learning. In Proceedings of the 2014 Conference on Interaction Design and Children (IDC ’14), 17–20 June 2014, Aarhus, Denmark (pp. 155–164). New York: ACM. https://dx.doi.org/10.1145/2593968.2593969
Zwass, V. (2010). Co-creation: Toward a taxonomy and an integrated research perspective. International Journal of Electronic Commerce, 15(1), 11–48. https://dx.doi.org/10.2753/jec1086-4415150101
Downloads
Published
How to Cite
Issue
Section
License
TEST
