Evaluating the Quality of the e-Learning Experience in Higher Education - PowerPoint PPT Presentation

Evaluating the quality of the e learning experience in higher education
Download
1 / 22

  • 419 Views
  • Uploaded on
  • Presentation posted in: General

Evaluating the Quality of the e-Learning Experience in Higher Education. Anne Jelfs and Keir Thorpe, Institute of Educational Technology (IET), The Open University. Introduction. Complexity of student experience Problems facing evaluators Different research methodologies

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Evaluating the Quality of the e-Learning Experience in Higher Education

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Evaluating the quality of the e learning experience in higher education

Evaluating the Quality of the e-Learning Experience in Higher Education

Anne Jelfs and Keir Thorpe,

Institute of Educational Technology (IET),

The Open University


Introduction

Introduction

  • Complexity of student experience

  • Problems facing evaluators

  • Different research methodologies

  • Audits promote evaluation

  • Holistic view – why? how? when? and what? of evaluation


Pressures

Pressures

  • Develop transferable student abilities

  • Pressure within the UK and worldwide

  • Governmental and employer pressure

  • Requirement – learners to participate in a networked information society

  • Emphasis on student-centred learning


Evaluators

Evaluators

  • Experienced evaluators needed in production, usability and effectiveness of online delivery

  • The Internet and ‘blended learning’ with ICT elements now widespread


Two sides of e learning

Promoted by government initiatives

Cost effectiveness of e-Learning

Permits tailored education

Can develop in ways which impoverish education

Time & production costs of development

Students & tutors lacking skills to use

Two Sides of e-Learning


The impact of quality assurance

The Impact of Quality Assurance

  • Parallel to increased use of ICT is rising interest in quality assurance strategies

  • Lighter QAA touch & emphasis on stakeholders – students, employers, funders, society

  • Emphasis on outcomes rather than processes

  • QAA looking for an institution’s strategy


Why evaluate the learner s experience

Why Evaluate the Learner’s Experience?

  • Subject review & institutional audits

  • Act as: educational ‘connoisseurs’ with approaches and tools to use in order to act as experts informing policy

  • Require: awareness and understanding of what is experienced

  • Require: personal judgement

  • Require: ability to articulate subtleties to a wider audience


Questions to ask yourself

Is the student experience exceptional?

Are there defects in the system or delivery?

Fit for purpose?

Value for money or adds costs?

Transformative/Value adding?

Students’ perceptions?

Effect on student numbers?

Questions to Ask yourself


Metrics for e learning evaluation

Metrics for e-Learning Evaluation

  • Efficiency of the teaching and learning process

  • Readability & usability of materials

  • Communication

  • Social relationships

  • Costs


Quality of the e learning experience

Quality of the e-Learning Experience

  • Efficiency of the teaching and learning process

  • Accurate student data

  • Drop out rates

  • Student assessment results

  • Student satisfaction ratings

  • Training of tutors


Quality of the e learning experience1

Quality of the e-Learning Experience

  • Efficiency of the teaching and learning process

To best understand the potential for

e-Learning:

  • Start with 1-2 courses

  • Is this course needed by students?

  • Is the assessment electronic?

  • Collection of computer log data


Evaluating the quality of the e learning experience in higher

Materials and Communication

  • 3. Communication

  • Communication with tutors & other students

  • Skills in using technology

  • Access from home/campus/3rd party

  • Group work

2. Readability & usability of materials

  • Peer review of materials

  • Reuse as learning objects

  • Online exercises

  • Study skills

  • e-Library and e-journal access


Evaluating the quality of the e learning experience in higher

Social Relationships and Costs

  • 5. Costs

  • Costs per student of recruitment

  • Training of tutors

  • Costs of electronic access

  • Reuse of learning materials & learning objects

4.Social Relationships

  • Hours working online

  • Student motivation

  • Student support & interaction with other students


Robson s 3 approaches to human computer interaction

Robson’s 3 Approaches to Human-Computer Interaction

  • Learning theory is dominant – how students and their interaction with computers fit together

  • How students learn when using computers

  • Using the technology to teach in ways that are novel and unique to the environment


How are we going to evaluate the e learning experience

How are we going to Evaluate the e-Learning Experience?

  • Evaluation goals need to be clearly defined

  • Is this a formative or summative evaluation?

  • How should the data be gathered?

  • Should quantitative or qualitative methods be used?


How are we going to evaluate the e learning experience1

How are we going to Evaluate the e-Learning Experience?

Methods include:

  • Surveys

  • Observations

  • Semi-structured interviews

  • Focus groups

  • Cognitive walk-throughs

  • Qualitative studies can provide examples of usability difficulties


How are we going to evaluate the e learning experience2

How are we going to Evaluate the e-Learning Experience?

Surveying students’ views:

  • detects feelings of overload

  • informs necessary course adjustments

    Survey methods:

  • paper based with

  • telephone supplementary interviews

  • increased use of electronic surveys

  • focus groups


When to evaluate the e learning experience

When to Evaluate the e-Learning Experience?

  • Dependent on whether the evaluation is formative or summative?

  • When is the student deemed to have completed the course?

  • Will formative evaluation during the course engage students more?

  • Harvesting evaluation information from student interaction with the course


What does the evaluation provide

What does the Evaluation provide?

  • Audio/video record of the computer screen, the user’s comments and keystrokes

    Baseline data to:

  • make adjustments, possibly ‘instant’ feedback

  • for external audit

  • shape e-learning strategy

  • input into associated resources – library and information literacy provision


Data protection

Data Protection

  • Students and tutors need to be kept informed about how the data they provide is being used

  • Students and tutors retain the right to remove what they have provided at any time

  • Evaluators have to ensure secure access to the data

  • Evaluators must get consent especially if using the data in a different context


Conclusion

Conclusion

  • The Open University is interested in the quality of the student experience

  • To improve that experience needs understanding of student and staff needs

  • Here we have aimed to provide a framework in which to conduct evaluations

  • Current work on modelling courses will provide data on points of concern that arise in blended and e-learning


Evaluating the quality of the e learning experience in higher

To praise & ask queries, contact Anne:

Contact

a.e.jelfs@open.ac.uk

To complain, contact Keir:

k.m.thorpe@open.ac.uk


  • Login