Learning Analytics: Looking to the Future
Simon Knight, Alyssa Wise, Xavier Ochoa, Arnon Hershkovitz
Editors, Journal of Learning Analytics
jla.editorial@gmail.com
ABSTRACT. This second issue of the Journal of Learning Analytics in 2017 is the first edited by the full new journal editorial team. As the baton is passed on, we would like to thank the founding editors for their work initiating the journal and nurturing its development over the past several years. We look forward to continuing that tradition of excellence. This issue includes four research paper contributions, and a special section on the ‘Shape of Educational Data’. This editorial is also an opportunity for us to reflect on the development of the journal so far, and describe some changes we are making to continue the expansion and maturation of a growing community of learning analytics researchers and practitioners.
Keywords: learning analytics, practice, practitioners, research, data
ISSN 1929-7750 (online). The Journal of Learning Analytics works under a Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0)
(2017). Learning analytics: Looking to the future. Journal of Learning Analytics, 4(2), 1–5. http://dx.doi.org/10.18608/jla.2017.42.1
1 FIRST, A LOOK TO THE PAST
The Journal of Learning Analytics was founded in 2014 as a venue to publish and disseminate the highest quality research relating to the opportunities and challenges of collecting, analyzing, reporting and mobilizing data traces with the specific intent to improve learning processes and outcomes. At the time of the journal’s founding, the Society for Learning Analytics Research (SoLAR) and Learning Analytics and Knowledge (LAK) Conference had been in existence a mere three years, but were already flourishing as a research and practice community due to the rapid growth of interest in learning analytics. At that time, the creation of the journal represented an important step in the development of the field, offering an additional peer-reviewed venue through which to share high quality learning analytics research and practice.
Since its founding, the Journal of Learning Analytics has published 10 issues in 4 volumes (three issues per volume, one volume per year with the current volume ongoing). These numbers include, in total, 124 works (75 research papers, 10 editorials, 12 special section editorials, 16 summer institute sparks, 5 practitioner papers, 5 article commentaries and 1 invited paper).
A total of 318 authors have contributed to the journal, with an average of 2 to 3 authors per paper. These authors belong to 22 different countries (Figure 1). While the author pool is predominantly from the Anglosphere (214 authors come from USA, Great Britain, Canada or Australia), there are representatives from most European countries, Latin America, Asia and Africa. Twenty-five percent (31) of the works published in the journal were the result of international cooperation.
Figure 1: Proportion of JLA authors by country. Size and color determine the contribution of that country to the author pool. The links represent international co-authorship.
According to values obtained from citations counts in Google Scholar, on average, each citable article published in JLA between 2014 and 2015 was cited 4.5 times in 2016. This estimate of the journal’s impact1 reflects the importance of JLA as a source of reference works in the field of Learning Analytics. The 5 most cited articles in 2016 are examples of seminal works in the field: "Early Alert of Academically At-Risk Students: An Open Source Analytics Initiative", "Epistemology, Assessment, Pedagogy: Where Learning Meets Analytics in the Middle Space", "Learning Analytics for Online Discussions: Embedded and Extracted Approaches", "Why Theory Matters More than Ever in the Age of Big Data" and "Understanding, Evaluating, and Supporting Self-Regulated Learning Using Learning Analytics".
Three and a half years later, we now reach another important inflection point in the growth of the field. SoLAR and LAK are each seven years old, JLA now moves into its fourth year, and 2017 welcome the first edition of the Handbook of Learning Analytics, a publication designed to consolidate and overview the research conducted thus far into a single easily-referenced volume (the Handbook is freely available as an open publication on the SoLAR website at https://solaresearch.org/hla-17). At this moment as well, the founding editors now pass on the baton. As the full new editorial team comes on board, we would like to take this opportunity to thank the founding editors for their efforts and dedication to initiate and develop the journal, as well as for their continued contributions to the field. Broad-based leadership and planned succession are a strong feature of all SoLAR endeavors, and as the incoming editorial team, we are honored and humbled to take on the charge of maintaining the journal’s tradition of excellence. In addition, we are excited for the new directions ahead as the field continues to grow. Below we outline both upcoming changes to the journal and existing features that increase in importance as the field matures.
2 INTRODUCING PRACTICAL REPORTS
To meet our goals of serving both the research and practitioner communities, and equally importantly fostering cross-talk and communication across them, we are pleased to introduce a new journal section devoted to Practical Reports. This section replaces the former Hot Spots section. The inclusion of Practical Reports will serve three purposes for the learning analytics community. First, it seeks to broaden the evidence base available through the rigorous reporting of efforts to implement learning analytics in real-world contexts and the practical issues that are encountered in doing so. Second, it seeks to expand the readership and impact of the JLA by providing relevant information to those interested in the questions involved in putting learning analytics into practice. Finally, we see it as part of a larger effort to increase communication between researchers and practitioners in the service of overall learning analytics excellence. We note that Practical Reports are conceived as a specific kind of contribution relating to practice rather than any contribution coming from practitioners. This is a deliberate choice as we believe that for the field to develop its impact, it is more important that work addressing both practical and research areas is reported than who conducts the work. Researchers who want to see their work used at scale may engage in efforts of practice that they will want to report, and there are already examples of practitioners publishing rigorous research on their learning analytics applications. Thus the new section refers to a particular type of contribution, not to whom is making it.
Practical reports will describe the application of learning analytics across a diversity of contexts and serve as case studies of authentic learning analytics applications that have relevance to the wider community. These reports will describe new or innovative learning analytics practices, programs, techniques or application in a specific context of practice. This may include efforts to apply learning analytics in pilot projects or in “at scale” implementations, efforts to evaluate learning analytics use in practice, efforts to develop institutional data repositories or pipelines, efforts to develop institutional policies or practices surrounding learning analytics use, and critical examinations of organizational challenges, tactics and strategies.
3 INTRODUCING TOOL & DATASET REPORTS
To build the community and its impact, the Journal of Learning Analytics is also now accepting papers that describe datasets and/or tools and their significance for the learning analytics community. The learning analytics field brings data and learning together; these new submission types recognise this in the journal by making data and the tools to analyze that data available, contextualised in learning environments of relevance to the learning analytics community. These papers are intended to foster collaboration and development of new approaches based on existing community work.
Dataset reports will typically introduce data that arises from actual learning processes and will frame it with theoretical foundations that allow for understanding of its context and its potential analysis. Such data can be drawn from a learning experience in any domain, in any learning setting, and with any population - all, of course, should be explicitly presented in the paper. Such data can be based on online or face-to-face settings. If relevant, complementary data (such as demographics, data from surveys, etc.) should also be provided in order to allow a rich understanding of the learning experience.
Tools reports will typically introduce novel tools and methods to analyze data in a way that may enable replication studies and extensions of existing analyzes to other learning settings. These reports should detail both the tool’s purpose and how to appropriately use it (including any assumptions that must be met). We also expect such papers to educate readers about the ways the presented tools might enrich exploration of data, for example by presenting a few case studies.
Both data and tools reports must include links to the data or tools described, preferably in openly available public repositories; if this is not the case, the report should describe procedures for requesting access. JLA does not offer hosting services for tools or data.
4 INTRODUCING BOOK REVIEWS
The growth of the field has also seen the emergence of several books on the topic. Thus the Journal of Learning Analytics is now pleased to consider proposals for book reviews. These reviews will inform readers about recent publications relevant to learning analytics and the use of big data in education by providing concise overviews of the book's' primary topics, themes, and messages paired with a critical analysis of its strengths, weaknesses, and relevance to the field. Book reviews are commissioned by the editors. Please contact the editors should you wish to propose a book to review or be reviewed.
5 RESEARCH SUBMISSIONS INCLUDE REVIEW PAPERS
The Research Section of the Journal of Learning Analytics continues to welcome papers that describe original empirical or theory-building research. The Journal of Learning Analytics has always welcomed papers that review the state of the art in a particular area of learning analytics, but as the field now matures, grows, and diversifies, such work becomes critical to maintain a shared common ground. Review papers offer rigorous examinations of relevant literature(s) in order to put forward a novel theory, framework, or even empirical result (e.g. via meta-analysis). We welcome these papers alongside the continued call for empirical and theoretical submissions to the journal’s research papers.
6 SPECIAL SECTION PROPOSALS ENCOURAGED
The Journal of Learning Analytics is now in its fourth volume of publication. Included in these published issues are a variety of special sections covering a range of topics such as Multimodal Analytics, Learning Analytics and Learning Theory, and Ethics and Privacy in Learning Analytics alongside special sections showcasing invited papers from the International Learning Analytics and Knowledge Conferences. The current issue includes a special section on the Shape of Educational Data. We continue to invite proposals for special sections, as described in the journal Focus and Scope via email to the editors. Archived ‘Calls for Papers’ can now be viewed in the announcements section on the journal website in order to provide a model for prospective guest editors.
7 STRUCTURED NOTES WILL BUILD CONNECTIONS BETWEEN RESEARCH AND PRACTICE
To build the conversation between research and practice across different community members engaged in learning analytic work, all Journal of Learning Analytics articles will now include ‘structured notes’ alongside the abstract. This change follows several other journals that have recently introduced bulleted “highlights” in addition to the abstract as a way to describe the significance of the work for particular stakeholders. For research papers, these “Notes for Practice” will highlight the significance of the research for practical contexts, including how the findings might create concrete implications for learning analytics implementations. For practical reports, the “Notes for Research” will highlight the significance of the work for researchers, including how the findings might contextualise existing findings or open up new avenues requiring research. These notes will comprise short bulleted overviews, following the structure indicated in the journal Author Guidelines pages.
8 THE CURRENT ISSUE
This issue of the journal includes four research papers, alongside a special section on ‘The Shape of Educational Data’ guested edited by Colleen Ganley and Sara Hart.
The research papers cover a diverse range of topics of relevance to the learning analytics community. Noting the potential for low-level features to support early intervention, Casey and Azcona discuss the use of keystroke data in developing a classifier for the early detection of poor performers. The papers by Saucerman, Ruis and Williamson Shaffer as well as that by Rau each focus on higher level cognitive features of learning through analyzing student discourse in order to understand ‘reflection-on-action’ and social mechanisms in visual concept-representation respectively. Finally, taking a systems approach to school leadership, Deakin Crick, Knight, and Barr introduce Hierarchical Process Modelling to support a holistic approach to data use for school improvement.
The special section includes outputs from the ‘Shape of Educational Data’ meeting, held in Fairfax, Virginia in April 2016. This event brought together interdisciplinary contributions drawing on mathematics, psychology, computer science, and learning analytics to inform our understanding of learning in online platforms based on the data generated by them. Three sets of articles discuss different online learning systems and their related data streams, present descriptions of topological data analysis and their application to modeling student learning, and describe the analysis of factors related to student learning based on data from online systems.
More information about the individual Shape of Educational Data papers is described in the Special Section Editorial. We hope you enjoy this issue.
_________________________
1 This metric was calculated using the same formula as the Thomspon Reuters’ Journal Citation Reports (JCR) Impact Factor, however the input datasets are not identical (JCR uses a more restricted dataset than that provided by Google Scholar). For this reason, this figure cannot be directly compared to JCR Impact Factors.