The University is investing in the use of learning analytics for course design, attainment, and improving the student experience.
The field of learning analytics along with its associated methods of online student data analysis holds great potential to address the challenges confronting educational institution and educational research. By merging technical methods for data mining and with current educational theory research and practice, learning analytics has provided novel and real-time approaches to assessing critical issues such as student progression and retention, establishment of indicators of 21st century skills acquisition, as well as personalised and adaptive learning.
The University of Edinburgh has a wide range of activities in the field of learning analytics. As shown in the diagram below, these activities cross many disciplinary, organisational, practice, and research boundaries. Led by the Vice-Principal Digital Education, Centre for Research in Digital Education, School of Informatics, Information Services, Student Systems, and the Institute for Academic Development, activities in learning analytics include University leaders, researchers, and practitioners from support, research, and academic units of the University collaborating on a variety of projects funded through both internal and external sources. Only some projects use data about the students from the University of Edinburgh. Other projects are fully research and do not have a direct impact on the students from the University of Edinburgh, though some of the findings and technologies may be used at the University of Edinburgh in the future. Although the university is involved in much collaboration with other universities, no data are shared.
Ethics and Privacy Protection
The University is committed to the ethical use of data and practices that are respectful of user privacy and compliant with the national and European legislation. Analytics at the University are exclusively used to understand and increase the success and learning experience of our students, enhance instruction capabilities of teaching staff, and inform institutional data making. The University takes an active role in national and international initiatives that support the ethical and privacy protective use of learning analytics, and all research activities in this area are carried out in accordance with the UK Research Integrity Office: Code of Practice for Research. The University’s active participation in the development of the Jisc’s Code of Practice for Learning Analytics is notable. Decisions about learning analytics at the University are made according to the practices recommended by the Jisc Code of Practice. When external organizations are contracted to provide learning analytics services, contracts are implemented to be in compliance with the relevant UK and European legislation regulating personal data use and processing. Data analysed are first anonymized with the state of the art procedures before shared with the contracting organizations.
Learning Analytics Activities
Student-focused activities local at the University of Edinburgh
This project asks: ‘How can University teaching teams develop critical and participatory approaches to educational data analysis?’ It seeks to develop ways of involving students as research partners and active participants in their own data collection and analysis, as well as foster critical understanding of the use of computational analysis in education. This work was funded by a Principal’s Teaching Award Scheme grant, and is working with students on specific courses within the Masters in Digital Education.
Information Services explored some Learning Analytics options within the Learn and Moodle virtual learning environments, working with a small number of specific courses. Projects and tools include those which allow students to see some of their own data and to help them understand their activity and learning patterns. Work in this space has concluded for the present, but these projects have provided valuable information about student attitudes to data and privacy which is being used to inform several of the other projects listed.
The University of Edinburgh is one of the pioneers in the space of massive open online courses. The researchers in Information Services, Centre for Research in Digital Education, School of Informatics, and Institute for Academic Development are actively engaged in analysis of digital trace, demographic and success data of the students who are enrolled into the MOOCs. The analysis to date involves understanding of the study patterns, effects of social networks on student success, and other demographic data on the success and experience of MOOC learners. The researchers from the University of Edinburgh are also actively collaborating with Technical University of Delft, Massachusetts Institute of Technology, University of Michigan, University of South Australia, University of Texas at Arlington, and University of Memphis.
Student-focused activities in collaboration with other institutions
The research on video analytics is conducted primarily in the collaboration with the University of South Australia, University of New South Wales, University of Sydney, and University of British Columbia. Analytics are developed to study effects of instructional conditions and experience on adoption of the video annotation software named Online Video Annotations for Learning (OVAL). Analytics are based on the use of digital traces of interaction with OVAL and used in the studies are conducted with students of performing arts and engineering and with faculty members for their academic development.
Flipped Classroom Analytics
The research on analytics in flipped classrooms is primarily focused on the development on the development of methods that allow for understanding the types of strategies and strategy changes learners follow throughout academic semester based on the analysis of digital traces recorded by VLEs. These analytics are used to inform improvement of instructional designs and advancement of learning experience. This research is in collaboration with the University of Sydney, University of South Australia, and University of Belgrade.
Supported by the European Association for Research on Learning and Instruction (EARLI) as a Centre for Innovative Research, the goal of this research is to develop measurements of students’ cognition, metacognition, emotion and motivation during learning in order to support the development of more powerful adaptive educational technologies. This research is conducted in collaboration with Radboud University Nijmegen, University of Oulu, North Carolina State University, Technische Universität München.
Learning dashboard effectiveness
This project works on the identification of common problems faced by teachers and students when learning online, and aims to determine the types of learning analytics teachers would find useful to effectively address these problems. The project is developing a web-based analytics tool named Loop that supports teachers to more easily interpret learning analytics to help them improve teaching and learning practices. This research is in collaboration with the University of Melbourne, University of South Australia, and Macquarie University.
Learning Beyond the LMS
As educators increasingly embrace social technologies to support learning, challenges arise in evaluating the quality and nature of student participation in activities using technology external to the institution's Learning Management System (LMS). This project extends the field of Learning Analytics (LA) by developing an open source toolkit for performing sophisticated analysis of learners' engagement in connected learning environments. This project is in collaboration with the University of Sydney, University of Texas at Arlington, University of South Australia, and University of Technology Sydney.
Institution-focused activities local at the University of Edinburgh
Teaching and learning dashboards
During 2015, Student Systems were given a steer from senior management to:
- Develop our use of student data to support ways to enhance learning & teaching, the student experience and operational effectiveness;
- Focus activity on what will make a difference at School level – provide support, help develop insights and share practice;
- Focus on the accessibility, visualisation and transparency of data, helping to simplify and manage complexity;
- Examine the use of dashboards to support these objectives.
Prototypes were developed in the second half of 2015 using both the BI and Qlikview tools and delivered to a number of forums with senior representatives from Schools and Colleges. The dashboards received consistent, positive enagement and feedback from the academic community.
Funding was secured to help move these dashboards from prototype to service for the 2016/17 academic year and the dashboards are now used to provide greater insight to Schools. These dashboards are complementary to the work being done to develop learning analytics for direct, individual student support and better course design (see above).
Learning Analytics Project with Civitas
We ran a 2 year pilot with Civitas Learning (a leading US company now expanding into the UK) using data from our fully online Masters level programmes and courses. A governance group was established to guide the project, under the leadership of our then VP Jeff Haywood, and reported to our Knowledge Strategy Committee and Learning & Teaching Committee.
The choice of the online Masters programmes as the pilot area was a critical one as it had the advantage of being a readily identifiable and isolatable pilot group that is large enough for the pilot scheme to work within. This was a data rich environment with strong student engagement in the digital learning environments.
This project allowed us to gain experience of developing learning analytics models, promoted teachers’ and students’ understanding of this area, improved our understanding of where areas of weakness exist in our data collection, and helped to develop a supporting Learning and Teaching Analytics Policy.
Institution-focused activities in collaboration with other institutions
To assist European universities to become more mature users and custodians of digital data about their students as they learn online, the SHEILA Project will build a policy development framework that promotes formative assessment and personalized learning, by taking advantage of direct engagement of stakeholders in the development process. It will run over 2016-18.
Learning analytics adoption
Although there is rapidly growing interest in learning analytics, there are limited resources that can inform institutions in how best to commence and deploy learning analytics. This presents a serious impediment as Universities seek to engage in learning analytics and build institutional capacity. The research focuses on how learning analytics are informing teaching practice, personalised learning, and applications for improving retention and identification of students at-risk. This research is conducted in collaboration with Macquarie University, The University of Melbourne, University of New England, University of Technology, Sydney, and University of the Sunshine Coast.
People Involved in Learning Analytics at the University of Edinburgh
- Professor Sian Bayne, Chair in Digital Education and Assistant Principal Digital Education
- Lisa Dawson, Director of Student Systems and Administration
- Professor Dragan Gasevic, Chair in Learning Analytics and Informatics (now at Monash University)
- Renate Gertz, University Data Protection Officer
- Melissa Highton, Director of Learning, Teaching and Web Services and Assistant Principal Online Learning
- Dr. Jeremy Knox, Lecturer of Digital Education
- Gavin McLachlan, Chief Information Officer and Librarian
- Hamish MacLeod, Senior Lecturer of Digital Education
- Anne-Marie Scott, Head of Digital Learning Applications and Media, Information Services
- Dr Yi-Shan Tsai, Research Associate, Centre for Research in Digital Education