The student experience of blended learning

1. The student experience of blended learning Dr. Rhona Sharpe rsharpe@brookes.ac.uk Oxford Centre for Staff and Learning Development Team: Greg Benfield George Roberts Richard Francis

2. “Besides all the complexity created by marked differences across subject areas and myriad individual differences among both staff and students which prevent simple patterns emerging, there are additional crucial differences between the idealized world described by research and the actual world experienced by the participants.” Entwistle, N., McCune, V. and Hounsell, J. (2002). 'Approaches to Studying and Perceptions of University Teaching-Learning Environments: Concepts, Measures and Preliminary Findings.' onlinehttp://www.ed.ac.uk/etl/docs/ETLreport1.pdf.

3. HEA review of undergraduate experiences of blended e-learning • How is the term ‘blended learning’ being used in higher education? • What are the underlying rationales being used for promoting blended learning? • What monitoring and evaluation strategies are being adopted for ensuring and enhancing the quality of blended learning? • What impact is blended learning having on the student experience? • What are the success factors for blended learning?

4. HEA review methodology • field work (interviews, document analysis) using the teams’ connections with practitioners and managers – to give access to private/grey literature • highlight the small number of transformative and thorough studies, making visible the existing research, policy and practice which is transforming the student experience. • review is based on real world practice of institutions and courses and actual world of students. • review will bring together the literature on student experience and institutional good practice to make recommendations for practitioners and managers.

5. Institutional good practice Student experience pedagogies technologies strategies/rationales staff dev evaluations QA processes holistic student exp learning designs student attributes support Two basic streams

6. Institutional visits

7. How is the term ‘blended learning’ being used? • Wide scale use of virtual learning environments to provide supplementary course resources • Radical, transformative course (re)designs to improve learning • A holistic view of technology, including use of learners’ own technologies to support learning

8. What are the underlying rationales being used for promoting blended learning? Institutional rationales were highly contextualised and specific to each institution: • Flexibility of provision • Supporting diversity • Enhancing the campus experience • Operating in a global context • Efficiency

9. What are the underlying rationales being used for promoting blended learning? Course level rationales mostly in response to practical challenges: • Loss of staff-student contact • Engaging students in out of class activities • Large classes • Inconsistency in quality and quantity of feedback • Developing professional skills

10. What monitoring and evaluation strategies are being used? The institutional VLE survey Range from • an item in the National Student Survey like ‘did you find the VLE useful to your studies?’ to (less frequently) • large, representative student (and staff) surveys involving questionnaires and focus groups that gather data on usage patterns and satisfaction ratings and discriminate between user groups (e.g., department, year, gender, etc.)

11. Institutional surveys of VLE use find: “We’ve never done any surveys, ever, that have given other than the students want more of it, wider and deeper” (Longside 2) Students value flexible access to course resources: • The ‘one stop shop’ • Access to lecture notes • Support for students with disabilities • Maintaining a connection with the institution Students are concerned about • Inconsistency in use between modules • Time and expense associated with downloading and printing

12. Some problems with surveys • Do not reveal ‘how’ or ‘with what’ technology helps students to learn • Do not differentiate between students with diverse backgrounds and needs • Implication of withdrawal of service • Lack of triangulation of data

13. Six of the seven institutions explicitly identified institutional level evaluation of blended learning as problematic, e.g.: • One of the things that’s been seriously lacking over the last five years at least, has been a lack of interest in evaluation, proper evaluation, not lip service evaluation like the student satisfaction surveys (Blackwater 2). • We don't have any systematic, institution-wide, sufficiently detailed research into the student experience in my view (Longside 1).

14. What monitoring and evaluation strategies are being used? • The institutional VLE survey • Business audits of VLE systems • Systematic module level feedback • Case studies linked to project funding

15. Business audits of VLE systems • E.g. KPMG auditors visit for a few days, produce a report “I think it is that notion of mission critical. People now regard it in terms of things like the risk assessment…. The VLE is now well up the list, because of the amount of stick we get if things fall over” (Eastonhall 1) • Know very little about this, including whether and/or what impact they have on blended learning development

16. What monitoring and evaluation strategies are being used? • The institutional VLE survey • Business audits of VLE systems • Systematic module level feedback • Case studies linked to project funding

17. Systematic module level feedback • Existing module feedback items need revising:“We just thought … we’ll just use our ordinary module evaluation. Well it wasn’t going to work was it? It’s not asking the right sort of questions.” (Deepshire 1) • ‘Systematic’ implies • Standardised module evaluation forms that include items on VLE use • Central analysis • Synthesis and reporting to departments, who incorporate findings into action plans

18. What monitoring and evaluation strategies are being used? • The institutional VLE survey • Business audits of VLE systems • Systematic module level feedback • Case studies linked to project funding

19. Case studies • The dominant method • Relies for effectiveness on variety of well-established internal dissemination avenues, including ‘informal’ practitioner networks • ‘Social’ repositories • Issues of accessibility to practitioners • Necessity for incentives to create and disseminate • E.g. Teaching Fellows, Ed Development unit, internal and external project funding, internal T&L conference, PGCert, institutional research strategy • Such elements are a ‘package’ at Longside

20. Mixed methods (triangulation) • Many evaluations cannot report improved student outcomes despite intentions to do so because of flawed methodology (Alexander 1999) • Mixed methods help evaluators learn not only what works (or doesn’t) but how and why it does (or doesn’t) • E.g. Boyle & Nicol (2003) evaluated introduction of PRS and used • 5 focus groups • Critical incident questionnaire • 365 item Likert scale survey • Staff focus group

21. Recommendations • Less frequent surveys aligned to institutional strategic planning • Purposive sampling • Mixed method data gathering • Audits of pedagogical approaches • Meaningful data about the impact on the student experience • Institutional systems for the dissemination of good practice

22. Critical success factors • There may be advantages to using a poorly defined term in institutional change • Institutional rationales which are contextualised and specific • Course designs or redesigns undertaken as a team, developed iteratively over a number of years in response to student feedback • Implementations which tackle real and relevant problems at the course level • Students’ understanding their own learning and the role of resources and technology in learning

23. References Alexander, S. (1999) An evaluation of innovative projects involving communication and information technology in higher education. Higher Education Research & Development. 18 (2), 173-183 Boyle, J. T. & Nicol, D. J. (2003) Using classroom communication systems to support interaction and discussion in large class settings. ALT-J Research in Learning Technology. 11(3), 43-57 Entwistle, N., McCune, V. and Hounsell, J. (2002). 'Approaches to Studying and Perceptions of University Teaching-Learning Environments: Concepts, Measures and Preliminary Findings.' onlinehttp://www.ed.ac.uk/etl/docs/ETLreport1.pdf