The oste objective structured technical examination for engineering students
This presentation is the property of its rightful owner.
Sponsored Links
1 / 26

The OSTE: Objective Structured Technical Examination for Engineering students PowerPoint PPT Presentation


  • 128 Views
  • Uploaded on
  • Presentation posted in: General

The OSTE: Objective Structured Technical Examination for Engineering students. Nandini Alinier, Guillaume Alinier University of Hertfordshire, U.K. ICEE 2005. Overview. Background & origins Application in an Engineering context Stages of the OSTE Development of stations

Download Presentation

The OSTE: Objective Structured Technical Examination for Engineering students

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The oste objective structured technical examination for engineering students

The OSTE: Objective Structured Technical Examination for Engineering students

Nandini Alinier, Guillaume Alinier

University of Hertfordshire, U.K.

ICEE 2005


Overview

Overview

  • Background & origins

  • Application in an Engineering context

  • Stages of the OSTE

    • Development of stations

    • Preparation for sessions

    • Pilot

    • First series

    • Second series

  • Analysis

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Background

Background

  • Objective Structured Clinical Examination (OSCE) (Harden 1975)

  • Originally developed in the mid-seventies in Dundee, Scotland

    • Used in healthcare education

    • Method to objectively assess the skills of healthcare trainees

  • Used in a project funded by the British Heart Foundation at UH during 2001-03 (Alinier et al 2004)

  • Perceived positively by students & lecturers (Alinier 2003)

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


The oste objective structured technical examination for engineering students

OSCE

  • Comprises 15 to 20 stations

  • Station: short task performed in 5 mins

    • Practical exercise

      • Requires an assessor

    • Theoretical exercise

      • Pen & paper

  • Gap period between stations (1 min)

    • Students rotate individually through the stations

    • Allows time for assessors to complete marking & reset any equipment

  • Timing of session by an electronic clock

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Osce continued

OSCE (continued)

  • Students given explicit written instructions

  • Objective marking sheet for each station

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Example of a station from an osce

Example of a station from an OSCE

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Applied to engineering

Applied to Engineering

  • Project undertaken in the School of Electronic, Communication & Electrical Engineering at the University of Hertfordshire, UK

  • Funded by the Higher Education Academy Engineering Subject Centre (Learning & Teaching Support Network(LTSN)Engineering)

  • Objective Structured Technical Examination (OSTE)

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Study about the oste

Study about the OSTE

  • Used for formative assessment

  • Piloted with level 2 BEng students

  • Conducted with level 1 BEng students (1st + 2nd semesters)

  • Each student did the OSTE twice

  • Circulated feedback questionnaires for students + assessors

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Stages of the oste

Stages of the OSTE

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Stage 1 development of oste stations

Stage 1: Development of OSTE stations

  • Gathering a variety of ideas about essential electrical engineering skills:

    • Seminar within School

    • Call for ideas via Email

  • Identified appropriate range of exercises doable in 5 mins

  • 16 stations were short-listed

  • Developed instruction sheets & marking criteria for each station

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Stage 1 development of oste stations continued

Stage 1: Development of OSTE stations (continued)

  • Types of stations:

    • Theoretical: pen & paper

    • Practical: assessor required

    • Computer-based: assessor required

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Stage 2 preparation for oste sessions

Stage 2: Preparation for OSTE sessions

  • Development of PC-based clock with an integrated voice prompt

  • Briefing of students

  • Training of staff to act as assessors

  • Designing of student & assessor feedback questionnaires

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Stage 3 pilot of oste

Stage 3: Pilot of OSTE

  • Group of 15 volunteer 2nd year BEng students

  • Resulted in changes in:

    • Tasks to be carried out

    • Marking sheet

    • Instruction sheet

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Example of a computer based station

Example of a computer-based station

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Stage 4 first series of oste

Stage 4: First series of OSTE

  • 1st year BEng students

  • During 1st semester (Dec 04)

  • Formative OSTE without feedback

  • To determine the level of students’ performance at the start of the course without preparing them too much for their 2nd OSTE

  • Feedback collected from students + assessors

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Stage 5 second series of oste

Stage 5: Second series of OSTE

  • Same group of 1st year students

  • During 2nd semester (Mar 05)

  • Run as “mixed-mode” OSTE:

    (Alinier 2003)

    • Collection of data about students’ performance

    • Feedback to students by assessor

  • Feedback collected from students + assessors

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Results students perspective

Results: students’ perspective

  • Out of 46 student feedback questionnaires

    • The OSTE was beneficial: 93.5%

    • The OSTE helped them developed confidence in using equipment: 73.9%

    • 52.2%: rated the OSTE as very good or good

      (5-point Likert Scale with 1=very good, mean=2.4 with 95% C.I. 1.4–3.4)

    • The OSTE should be used as a summative assessment tool: 28.3%

    • Number of times OSTE should be repeated per year: 2.9 (95% C.I. 1.4 - 4.4)

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Results assessors perspective

Results: assessors’ perspective

  • Out of 33 assessor feedback questionnaires

    • The OSTE should be incorporated into the 1st year BEng curriculum: 93.1%

    • The OSTE helped students develop their confidence in using equipment they were exposed to: 75.8%

    • 90.6%: rated the OSTE as very good or good

      (5-point Likert Scale with 1=very good, mean=1.5 with 95% C.I. 0.8 – 2.2)

    • The OSTE should be used as a summative assessment tool: 75.9%

    • Number of times OSTE should be repeated per year: 2.2 (95% C.I. 1.4 - 3.0)

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Further analysis

Further analysis

  • Identify key weaknesses among students

  • Any correlation between performance in OSTE and performance as per in-class assessment

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Oste results

OSTE results

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Conclusions

Conclusions

  • OSTE: perceived positively by students and assessors

  • Formative assessment 

  • Summative assessment ?

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Thanks for your attention

Thanks for your attention

Any questions?

[email protected]

[email protected]

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


References

References

  • Harden R M, Stevenson M, Downie W W, Wilson G M, "Assessment of clinical competence using an objective structured clinical examination", Medical Education, Vol 1, 1975, 447-451.

  • Alinier G, "Nursing students’ and lecturers’ perspectives of OSCE incorporating simulation", Nurse Education Today, Vol 23, No 6, 2003, 419-426.

  • Alinier G, Hunt W B, Gordon R, "Determining the effect of realistic simulation in nurse education: Study design and initial results", Nurse Education in Practice, Vol 4, No 3, 2004, 200-207.

  • Ross M, Carroll G, Knight J, Chamberlain M, Fothergill-Boubonnais F, Linton J, "Using the OSCE to measure clinical skills performance in nursing", Journal of Advanced Nursing, Vol 13, 1988, 45-56.

  • Hulett S, Gilder N, "The application and development of the Objective Structured Practical Examination in evaluating physiotherapy students’ performance at the University of Cape Town", South African Journal of Physiotherapy, Vol 42, No 2, 1986, 40-43.

  • Marshall G, Harris P, "A study of the role of an objective structured clinical examination (OSCE) in assessing clinical competence in third year student radiographers", Radiography, Vol 6, 2000, 117-122.

  • Mossey P A, Newton J P, Stirrups D R, "Scope of the OSCE in the assessment of clinical skills in dentistry", British Dental Journal, Vol 190, No 6, 2001, 323-326.

  • Townsend A H, Mcllvenny S, Miller C J, Dunn E V, "The use of an objective structured clinical examination (OSCE) for formative and summative assessment in a general practice clinical attachment and its relationship to final medical school examination performance", Medical Education, Vol 35, 2001, 841-846.

  • Harden R M, "Twelve tips for organising an OSCE", Medical Teacher, Vol 12, No 3/4, 1990, 259-264.

  • Sloan D A, Donnelly M B, Schwarts R W, Strodel W E, “The Objective Structured Clinical Examination: The new gold standard for evaluating postgraduate clinical performance”, Annals of Surgery, Vol 222, No 6, 1995, 735-742.

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


  • Login