Using Pedagogy and Learning Analytics to Manage Our Students

1. Using Pedagogy and Learning Analytics to Manage Our Students Neil GordonDepartment of Computer ScienceUniversity of Hull Higher Education AcademySTEM Annual Conference 2014University of Edinburgh

2. Overview • Pedagogy – the how we teach –provides the primary way to direct student behaviours • Learning Analytics is of growing interest within the context of Technology Enhanced Learning and can provide data on student activity • We will consider how these two ideas can be brought together to provide a way to manage student engagement

3. Developing activities and assessments for engagement • A variety of pedagogic approaches • For example: • Handing out material (esp. formative or summative examples) in lectures or other sessions • Incorporating student activities in lectures and formal contact meetings (including flipped lectures) • Some approaches are behavioural rather than pedagogic – e.g. rewarding attendance / penalising non-attendance

4. Formative versus Summative assessment • Utilise and adapting assessment to encourage regular and early engagement • Consider formative assessment – will they do it, compared to • Summative – they do it for marks • Balance over-assessment versus having sufficient data to potentially identify behaviours

5. What is learning analytics • Big Data is a popular topics in Computer Science and more general society – referring to the collation and use of large sets of data • Learning Analytics refers to the use of data from learning systems – typically a Virtual Learning Environment (VLE), though it could be from any system used for learning

6. So what can learning analytics give us?

7. Analytics can tell us WHAT students are doing Most active resource

8. WHICH resources they want • Most downloaded file • Some tools (e.g. many lecture capture systems) can show which part of a file / media is being accessed to help identify “difficult” topics

9. When they are doing it • Different analytics tools can offer analysis of when students access resources • Common to get spikes near assessment periods • Some tools offer time of day so it becomes possible to appreciate how and when students are using the learning resources

10. Who is (not) doing it • Tools can enable staff to check if students are accessing resources • Carrying out tasks – assessed or otherwise • And whether they are contributing to activities e.g. team activities • Web 2.0 and learning tools support this – with date and author stamping

11. How: student response to assessments

12. Team Work and analytics Can provide audit trails andcollect other data fromaccess logs

13. Engagement and attendance • What is engagement:not clearly defined. In educational terms, that students attend (possibly), read, prepare and carry out activities, and demonstrate attainment (through some form of testing) • Does attendance matter?The historical approach of reading for a degree would put the onus on students to choose how (even whether) to attend • Expectation that increased fees (England) would improve attendance rates – though data is patchy • But institutions are adopting more stringent and strict attendance monitoring regimes

14. Monitoring attendance • Range of empirical and experiential data show that attendance and performance are in correlation – though some outliers

15. Working to deadlines means missed deadlines • Understanding that (some) students work to last minute deadlines can help in teaching and supporting them • E-submission can lead to increased late penalties (until students understand the approach) • Suitable programme/module planning can help e.g. formative or low-risk (i.e. low contribution) assessments that let students practice submission E.g. submission deadline 4pm 7 Nov 2013.10% of the class received a penalty for late submission (within 1 day of original deadline) Late

16. Rules and Regs and engagement • Can use the stick approach • Attendance requirements: • Mandatory attendance • Registers • Follow up non-attendance • Penalty for non-attendance • Submission requirements • On time • Penalties for late submission • Penalties for not following guidance • But do these improve student engagement, or just student presence

17. So a model for encouraging engagement, supported by learning analytics • Initial formative activities (that can be logged/tracked) • Encourage attendance – “light warnings” for absence • Periodic summative assessments (computer tests or other) to check progress (for students and staff) • Combine results from attendance and engagement to target support • Adapt teaching based on topics students find difficulty with – identified from access logs and tests

18. Support mechanisms • Personal Tutors • Specific learning and study support • Personalised (targeted) support • E.g. Your mark so far is below 40%. You are advised to see X in order…. • Potential for relative performance (common overseas, seems less so in the UK): you are in the top 60% of the cohort…

19. Results • Combining attendance data with data from assessments enables students to be flagged as “at risk” • Better analysis if done across modules – combining attendance data with engagement data (formative and summative assessments, accessing material) • Can highlight “at risk” students • Allows the targeting of support

20. Conclusions: future work • Need to balance “big brother” with helpful teacher • MOOCS are offering a huge level of data on learners and their behaviour – suspected as one of the reasons why some institutions are offering them • Understanding learner behaviour may enable more suitable (flexible) approaches to how we teach • VLEs already support some aspects of this – next generation tools should take this further. • So what do you want from Learning Analytics? END