Reviewing Course Evaluation Data: Perception vs reality
This presentation is the property of its rightful owner.
Sponsored Links
1 / 22

Reviewing Course Evaluation Data: Perception vs reality Peter D Munn and Sheila D Scutter PowerPoint PPT Presentation


  • 70 Views
  • Uploaded on
  • Presentation posted in: General

Reviewing Course Evaluation Data: Perception vs reality Peter D Munn and Sheila D Scutter. Introduction. Graduate Course Evaluation Questionnaire (GCEQ) provided to all graduates Australia wide Comprises 25 Likert scale statements and two free response items

Download Presentation

Reviewing Course Evaluation Data: Perception vs reality Peter D Munn and Sheila D Scutter

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Reviewing course evaluation data perception vs reality peter d munn and sheila d scutter

Reviewing Course Evaluation Data: Perception vs reality

Peter D Munn and

Sheila D Scutter


Introduction

Introduction

  • Graduate Course Evaluation Questionnaire (GCEQ) provided to all graduates Australia wide

    • Comprises 25 Likert scale statements and two free response items

    • Six key areas of graduates’ experience

      • Clear goals and standards

      • Appropriate workload

      • Appropriate assessment

      • Good teaching

      • Generic skills

      • Overall satisfaction


Introduction1

Introduction

  • University of South Australia uses good teaching, generic skills and overall satisfaction as key performance indicators of perceived teaching quality

  • Areas of concern with Nursing program GCEQ scores.

    • Assumed to be due to course content

    • A comprehensive review process to understand specific areas of concern in nursing.


Nursing program

Nursing program

  • Three year undergraduate program

  • Offered in 2 city and 1 regional campus

  • Offered in internal and external mode

  • Approximately 1000 students

  • Many part time students

  • Access via Year 12, STAT, bridging programs

  • Special entry test for ATSI students

  • Very high participation of equity groups.


Method

Method

  • Graduate Course Evaluation Questionnaire(GCEQ)

    • GCEQ data for the nursing programs across the city and regional campuses were analysed by external and internal mode of study

    • Thematic analysis on comments on the ‘best aspects’ and ‘areas most in need of improvement’ responses analysed by mode of study

    • Individual item scores reviewed


Reviewing course evaluation data perception vs reality peter d munn and sheila d scutter

  • Student experience questionnaire (SEQ)

    • Distributed online to all currently enrolled students

    • Feedback on aspects of academic life, resources and services

    • Sixteen Likert scale items and two open response items concerning course and program quality

    • Responses compared by mode of study

    • Thematic analysis of comments by mode of study


Reviewing course evaluation data perception vs reality peter d munn and sheila d scutter

  • Course Evaluation Instrument

    • Each course in the University is evaluated every time it is offered

    • Course Evaluation Questionaire (CEI), online instrument developed by the University

    • CEI contains 10 core Likert-scale questions concerning course quality

    • Additional items may be added, this analysis concentrated on the 10 core items


Reviewing course evaluation data perception vs reality peter d munn and sheila d scutter

  • Focus Groups with Students

    Three focus groups were conducted:

    • Two with interstate, rural and metropolitan South Australian students

    • One with indigenous students and their learning support coordinator

  • Focus Groups with Staff

    • Eight staff representing both campuses


Results

Results

GCEQ Scores, Good Teaching


Results1

Results

GCEQ data

  • Concerns over good teaching, overall satisfaction and generic skills;

  • Low response rates may be a source of bias


Results2

Results

Student Evaluation Questionnaire

  • Responses to most statements were positive with a mean score over 4 (maximum 5);

  • Lower scores received for items relating to:

    • Support provided by teaching staff;

    • Timeframe for return of assignments;

    • Knowing what is expected in assignments;

    • Consistency in marking.


Results3

Results

CEI data

  • Limited response by students made interpretation of data difficult;

  • Feedback on courses ranged from very good to very poor;

  • Main areas of concern:

    • I felt there was a genuine interest in my learning needs and progress;

    • The workload for this course was reasonable given my other study commitments;

    • I have received feedback that is constructive and helpful.


Reviewing course evaluation data perception vs reality peter d munn and sheila d scutter

Focus Groups: GCEQ/SEQ analysis

  • Course Materials and Content

    • Little comment

    • Valued flexibility

    • Often arrived late

    • External students felt “second best

    • Revised courses in 2004 to update content


Reviewing course evaluation data perception vs reality peter d munn and sheila d scutter

  • Communication between students and academic staff

    • A major concern of external students

    • Interpreted as lack of interest by academics

    • External students felt disadvantaged compared to internal students.

    • Staff concerns about students accessing email

    • Geographical support groups disestablished


Reviewing course evaluation data perception vs reality peter d munn and sheila d scutter

  • Learning Support Service Availability

    • Requirements for assignment writing

    • Particular concern for indigenous students

    • Available online but prefer alternative delivery

    • Need for support/study groups

    • Difficulty accessing learning advisers.

  • Teaching and Learning Issues

    • Time not spent productively in workshops

    • Clinical experiences very positive

    • Staff/student ratio low

    • Lack of training for staff in external delivery

    • Student preparation for the online environment

    • High percentage for exams in science based courses


Reviewing course evaluation data perception vs reality peter d munn and sheila d scutter

  • Assignments and Assessment

    • Consistency of requirements within courses

    • Turnaround times major concern

    • Clarity of expectations.

    • Large number of students affecting turnaround


Discussion

Discussion

  • GCEQ scores indicated concerns about overall satisfaction, good teaching and generic skills

    • The instrument

    • Response rates

    • Timing

    • Interpreting the data

  • Students are not given the opportunity to provide many areas of this feedback in GCEQ.

  • Where is was possible to provide comment in the GCEQ, it was not reflected in item scores.


Discussion1

Discussion

Key concerns of students identified from thematic analysis and focus groups:

  • Timely, consistent and useful feedback;

  • Communication with academic staff;

  • Learning support;

  • Practical workshops/professional placements

  • Lack of preparation for study


Recommendations

Recommendations

Agreed practice model to be implemented

  • Provide up to date study guide

  • Early arrival of course materials

  • Fortnightly email communication

  • Respond to emails and calls in 24 hours

  • Contribute to discussion board weekly

  • Harvard reference material

  • Geographical location maps

  • Information packages for Maths and English support


Agreed practice model

Agreed Practice model…

Key points sheet for assignments and marking

  • Moderation of assignment marking

  • Staff development for external mode

  • Two week turnaournd for first assignment

  • Feedback before next assignment due

  • Promote course evaluation

  • Respond to student feedback


Reviewing course evaluation data perception vs reality peter d munn and sheila d scutter

  • Initiatives around GCEQ response rates and distribution.

  • Extension of preparatory courses (Sciences)

  • Review nursing workshops

  • Review assessment


Conclusion

Conclusion

  • GCEQ scores alone do not provide direction for program improvement.

  • Thorough analysis of course and program feedback is necessary to understand problems and to develop appropriate changes to programs.


  • Login