Standards quality assurance best practice and benchmarking in e learning
1 / 30

Standards, quality assurance, best practice and benchmarking in e-learning - PowerPoint PPT Presentation

  • Uploaded on

Standards, quality assurance, best practice and benchmarking in e-learning. Professor Paul Bacsich Matic Media Ltd, and Middlesex University, UK. The Menu. Standards (technical) Quality Assurance Standards (content) Standards (pedagogy and process) Best Practice Excellence?

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Standards, quality assurance, best practice and benchmarking in e-learning' - lynn

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Standards quality assurance best practice and benchmarking in e learning

Standards, quality assurance, best practice and benchmarking in e-learning

Professor Paul Bacsich

Matic Media Ltd, and

Middlesex University, UK

The menu
The Menu

  • Standards (technical)

  • Quality Assurance

    • Standards (content)

    • Standards (pedagogy and process)

  • Best Practice

    • Excellence?

  • Benchmarking

  • Conclusions

Standards technical
Standards (technical)

  • UK follows mainly IMS

  • Agency called CETIS set up by JISC to advise universities and colleges on IMS

  • A few mega universities (OU, Ufi, etc) are direct members of IMS

  • IMS Learning Design gaining influence

  • Also e-portfolios

Standards content
Standards (Content)

  • Quality Assurance Agency has set up “subject benchmarks”

  • More about generalised competences than detailed syllabi

  • See

Standards pedagogy and process
Standards (pedagogy and process)

  • Quality Assurance Agency (QAA)

  • “Code of practice for the assurance of academic quality and standards in higher education”

  • See

  • Not much on pedagogy – this is left to the discretion of the professor

Qaa in e learning
QAA in e-learning

  • Little has been done specifically on e-learning – but see…

  • “Collaborative provision and flexible and distributed learning (including e-learning)”

  • Recent – September 2004

  • Some feel it says too little, others do not want to be restricted

Digression on pedagogy
Digression on Pedagogy

  • Higher Education Academy

  • “works with universities and colleges, discipline groups, individual staff and organisations to help them deliver the best possible learning experience for all students”

  • Runs Subject Centres for each subject

  • Beginning to advise on e-learning

Best practice in e learning
Best practice in e-learning

  • Not much studied in the UK yet

  • OU a major source of advice

  • UKeU set up to crystallise best practice into an operational business

  • It failed – but its legacy may help

    • Committee for Academic Quality

  • US much more active – see e.g. “Quality on the Line” (IHEP, 2000)

In uk universities compete and now in e learning
In UK, universities compete- and now in e-learning

  • Universities want to judge how well they are doing in e-learning

  • And funding agencies also want to know

  • But universities don’t want to tell if they are doing badly! Not the public, not the funding agencies.

  • And universities (like people) are not good at judging themselves.


  • Like Activity Based Costing, it has been around for many years

  • Unlike ABC, but like BPR, quality, excellence, etc; no one is now sure what it means…

Back to basics xerox
Back to Basics (Xerox)

a process of self-evaluation and self-improvement

through the systematic and collaborative comparison of practice [process]

and performance [metrics, KPIs]

with competitors [or comparators]

in order to identify own strengths and weaknesses,

and learn how to adapt and improve

as conditions change.

Benchmarking dichotomies





Inputs or Processes



Collaborative [clubs]





(After Jackson)

Benchmarking Dichotomies

Focus of my work
Focus of my work

  • Focussed purely on e-learning

  • But not to any particular style (e.g. DL)

  • Oriented to institutions past the “a few projects” stage

  • Suitable for desk research as well as invasive studies

  • Suitable for single- and multi-institution studies

Benchmarking in universities
Benchmarking (in Universities)

  • There are several reports that will tell you how to do benchmarking in general

    • Higher Education Academy (UK)

    • Learning and Skills Development Agency (UK)

    • Department of Education Training and Youth Affairs (Australia)

Benchmarking in universities1
Benchmarking (in Universities)

  • And some agencies can help:

    • European Benchmarking Programme on University Management (ESMU, Brussels)

    • English Universities Benchmarking Club

Benchmarking in e learning
Benchmarking in e-Learning

  • There are veryfew reports

    • National Learning Network (UK) –not for universities, but for colleges

    • E-Learning Maturity Model (NZ) – brand new!

Quality best practice in e learning
Quality/Best Practice in e-Learning

  • There are a few reports (US):

    • APQC/SHEEO Study 1998 (US)

    • IHEP “Quality on the Line” 2000 (US)

  • And several projects (EU):

    • BENVIC


    • Swiss Virtual Campus @ Lugano: MINE

Excellence in e learning
Excellence (?) in e-Learning

  • New project: E-xcellence (EADTU and others)

  • Outside e-learning, several projects:

    • Consortium for Excellence in Higher Education (UK)

Benchmarking e learning

Benchmarking e-learning

A “synthesis”

Processes or outputs
Processes or Outputs?

  • Outputs first (can be done by desk research)

  • Processes later (best done in clubs or invasive studies)

  • Inputs not of interest to students; but of course of interest to funders

Metrics or bureaucratic
Metrics or Bureaucratic

  • Use a 6-point scale

    • 5 from Likert plus 1 more for “excellence”

  • Backed up by metrics where possible

  • Also contextualised by narrative

  • Remember the problems of judging “best practice”; judging “better practice” is easier

Other decisions
Other Decisions

  • Explicit (otherwise you are not trying)

  • Independent or collaborative

  • Internal or external

  • Horizontal: focus on processes across the whole institution; do not be seduced into individual projects

How many benchmarks
How Many Benchmarks?

  • It is like ABC: how many activities?

  • Answer: Not 5, not 500.

  • Better answer: Well under 100.

    • Composite some criteria together

    • Remove any not specific to e-learning

    • Be careful about any which are not provably critical success factors.

How many do others have
How Many do Others Have?

  • LSDA (UK) has 14

  • IHEP (US) has 24

  • APQC/SHEEO (US) had 14

  • (Breaking news) EMM (NZ) has 43

Pick and mix system
Pick and Mix System

  • 25 criteria (liable to grow to around 30)

  • 6 levels, backed up by qualitative and numeric information

  • Student-oriented

  • Focussed on critical success factors

  • Requires no long training course to understand, if you know about e-learning

  • Methodology-agnostic

Adoption phase rogers
“Adoption phase” (Rogers)

  • Innovators only

  • Early adopters taking it up

  • Early adopters adopted; early majority taking it up

  • Early majority adopted; late majority taking it up

  • All taken up except laggards, who are now taking it up (or retiring or leaving)

  • First wave embedded, second wave under way (e.g. m-learning after e-learning)


  • No systematic training for e-learning

  • Some systematic training, e.g. in some projects and departments

  • U-wide training programme but little monitoring of attendance or encouragement to go

  • U-wide training programme, monitored and incentivised

  • All staff trained in VLE use, training appropriate to job type – and retrained when needed

  • Staff increasingly keep themselves up to date in a “just in time, just for me” fashion except in situations of discontinuous change

Next steps
Next Steps

  • Correlate with “quality” and “excellence” projects in EU

  • Publish a review report on UK Committee for Academic Quality (in e-Learnng) August

  • Review underpinning methodologies (CMM etc)

  • Literature search outside Europe, US and Commonwealth

  • Series of workshops

    • at ALT-C 2005 Manchester September

    • At ACODE Australia November

    • at Online Educa Berlin December

Thank you for listening any questions

Thank you for listeningAny questions?

Professor Paul Bacsich

Global Campus, Middlesex University