1 / 30

Standards, quality assurance, best practice and benchmarking in e-learning

Standards, quality assurance, best practice and benchmarking in e-learning. Professor Paul Bacsich Matic Media Ltd, and Middlesex University, UK. The Menu. Standards (technical) Quality Assurance Standards (content) Standards (pedagogy and process) Best Practice Excellence?

lynn
Download Presentation

Standards, quality assurance, best practice and benchmarking in e-learning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Standards, quality assurance, best practice and benchmarking in e-learning Professor Paul Bacsich Matic Media Ltd, and Middlesex University, UK

  2. The Menu • Standards (technical) • Quality Assurance • Standards (content) • Standards (pedagogy and process) • Best Practice • Excellence? • Benchmarking • Conclusions

  3. Standards (technical) • UK follows mainly IMS • Agency called CETIS set up by JISC to advise universities and colleges on IMS • A few mega universities (OU, Ufi, etc) are direct members of IMS • IMS Learning Design gaining influence • Also e-portfolios

  4. Standards (Content) • Quality Assurance Agency has set up “subject benchmarks” • More about generalised competences than detailed syllabi • See www.qaa.ac.uk/academicinfrastructure/benchmark/

  5. Standards (pedagogy and process) • Quality Assurance Agency (QAA) • “Code of practice for the assurance of academic quality and standards in higher education” • See www.qaa.ac.uk/academicinfrastructure/codeOfPractice/ • Not much on pedagogy – this is left to the discretion of the professor

  6. QAA in e-learning • Little has been done specifically on e-learning – but see… • “Collaborative provision and flexible and distributed learning (including e-learning)” • Recent – September 2004 • Some feel it says too little, others do not want to be restricted

  7. Digression on Pedagogy • Higher Education Academy • “works with universities and colleges, discipline groups, individual staff and organisations to help them deliver the best possible learning experience for all students” • Runs Subject Centres for each subject • Beginning to advise on e-learning

  8. Best practice in e-learning • Not much studied in the UK yet • OU a major source of advice • UKeU set up to crystallise best practice into an operational business • It failed – but its legacy may help • Committee for Academic Quality • US much more active – see e.g. “Quality on the Line” (IHEP, 2000)

  9. In UK, universities compete- and now in e-learning • Universities want to judge how well they are doing in e-learning • And funding agencies also want to know • But universities don’t want to tell if they are doing badly! Not the public, not the funding agencies. • And universities (like people) are not good at judging themselves.

  10. Benchmarking • Like Activity Based Costing, it has been around for many years • Unlike ABC, but like BPR, quality, excellence, etc; no one is now sure what it means…

  11. Back to Basics (Xerox) a process of self-evaluation and self-improvement through the systematic and collaborative comparison of practice [process] and performance [metrics, KPIs] with competitors [or comparators] in order to identify own strengths and weaknesses, and learn how to adapt and improve as conditions change.

  12. Implicit Independent Internal Vertical Inputs or Processes Metric Explicit Collaborative [clubs] External Horizontal Outputs Qualitative (After Jackson) Benchmarking Dichotomies

  13. Focus of my work • Focussed purely on e-learning • But not to any particular style (e.g. DL) • Oriented to institutions past the “a few projects” stage • Suitable for desk research as well as invasive studies • Suitable for single- and multi-institution studies

  14. Benchmarking (in Universities) • There are several reports that will tell you how to do benchmarking in general • Higher Education Academy (UK) • Learning and Skills Development Agency (UK) • Department of Education Training and Youth Affairs (Australia)

  15. Benchmarking (in Universities) • And some agencies can help: • European Benchmarking Programme on University Management (ESMU, Brussels) • English Universities Benchmarking Club

  16. Benchmarking in e-Learning • There are veryfew reports • National Learning Network (UK) –not for universities, but for colleges • E-Learning Maturity Model (NZ) – brand new!

  17. Quality/Best Practice in e-Learning • There are a few reports (US): • APQC/SHEEO Study 1998 (US) • IHEP “Quality on the Line” 2000 (US) • And several projects (EU): • BENVIC • SEEQUEL • Swiss Virtual Campus @ Lugano: MINE

  18. Excellence (?) in e-Learning • New project: E-xcellence (EADTU and others) • Outside e-learning, several projects: • Consortium for Excellence in Higher Education (UK)

  19. Benchmarking e-learning A “synthesis”

  20. Processes or Outputs? • Outputs first (can be done by desk research) • Processes later (best done in clubs or invasive studies) • Inputs not of interest to students; but of course of interest to funders

  21. Metrics or Bureaucratic • Use a 6-point scale • 5 from Likert plus 1 more for “excellence” • Backed up by metrics where possible • Also contextualised by narrative • Remember the problems of judging “best practice”; judging “better practice” is easier

  22. Other Decisions • Explicit (otherwise you are not trying) • Independent or collaborative • Internal or external • Horizontal: focus on processes across the whole institution; do not be seduced into individual projects

  23. How Many Benchmarks? • It is like ABC: how many activities? • Answer: Not 5, not 500. • Better answer: Well under 100. • Composite some criteria together • Remove any not specific to e-learning • Be careful about any which are not provably critical success factors.

  24. How Many do Others Have? • LSDA (UK) has 14 • IHEP (US) has 24 • APQC/SHEEO (US) had 14 • (Breaking news) EMM (NZ) has 43

  25. Pick and Mix System • 25 criteria (liable to grow to around 30) • 6 levels, backed up by qualitative and numeric information • Student-oriented • Focussed on critical success factors • Requires no long training course to understand, if you know about e-learning • Methodology-agnostic

  26. “Adoption phase” (Rogers) • Innovators only • Early adopters taking it up • Early adopters adopted; early majority taking it up • Early majority adopted; late majority taking it up • All taken up except laggards, who are now taking it up (or retiring or leaving) • First wave embedded, second wave under way (e.g. m-learning after e-learning)

  27. “Training” • No systematic training for e-learning • Some systematic training, e.g. in some projects and departments • U-wide training programme but little monitoring of attendance or encouragement to go • U-wide training programme, monitored and incentivised • All staff trained in VLE use, training appropriate to job type – and retrained when needed • Staff increasingly keep themselves up to date in a “just in time, just for me” fashion except in situations of discontinuous change

  28. What’s next?

  29. Next Steps • Correlate with “quality” and “excellence” projects in EU • Publish a review report on UK Committee for Academic Quality (in e-Learnng) August • Review underpinning methodologies (CMM etc) • Literature search outside Europe, US and Commonwealth • Series of workshops • at ALT-C 2005 Manchester September • At ACODE Australia November • at Online Educa Berlin December

  30. Thank you for listeningAny questions? Professor Paul Bacsich Global Campus, Middlesex University p.bacsich@mdx.ac.uk www.cs.mdx.ac.uk/staff/profiles/p_bacsich.html

More Related