The oste objective structured technical examination for engineering students
Download
1 / 26

The OSTE: Objective Structured Technical Examination for Engineering students - PowerPoint PPT Presentation


  • 182 Views
  • Uploaded on

The OSTE: Objective Structured Technical Examination for Engineering students. Nandini Alinier, Guillaume Alinier University of Hertfordshire, U.K. ICEE 2005. Overview. Background & origins Application in an Engineering context Stages of the OSTE Development of stations

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' The OSTE: Objective Structured Technical Examination for Engineering students' - orenda


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
The oste objective structured technical examination for engineering students

The OSTE: Objective Structured Technical Examination for Engineering students

Nandini Alinier, Guillaume Alinier

University of Hertfordshire, U.K.

ICEE 2005


Overview
Overview Engineering students

  • Background & origins

  • Application in an Engineering context

  • Stages of the OSTE

    • Development of stations

    • Preparation for sessions

    • Pilot

    • First series

    • Second series

  • Analysis

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Background
Background Engineering students

  • Objective Structured Clinical Examination (OSCE) (Harden 1975)

  • Originally developed in the mid-seventies in Dundee, Scotland

    • Used in healthcare education

    • Method to objectively assess the skills of healthcare trainees

  • Used in a project funded by the British Heart Foundation at UH during 2001-03 (Alinier et al 2004)

  • Perceived positively by students & lecturers (Alinier 2003)

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


OSCE Engineering students

  • Comprises 15 to 20 stations

  • Station: short task performed in 5 mins

    • Practical exercise

      • Requires an assessor

    • Theoretical exercise

      • Pen & paper

  • Gap period between stations (1 min)

    • Students rotate individually through the stations

    • Allows time for assessors to complete marking & reset any equipment

  • Timing of session by an electronic clock

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Osce continued
OSCE (continued) Engineering students

  • Students given explicit written instructions

  • Objective marking sheet for each station

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Example of a station from an osce
Example of a station from an OSCE Engineering students

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Applied to engineering
Applied to Engineering Engineering students

  • Project undertaken in the School of Electronic, Communication & Electrical Engineering at the University of Hertfordshire, UK

  • Funded by the Higher Education Academy Engineering Subject Centre (Learning & Teaching Support Network(LTSN)Engineering)

  • Objective Structured Technical Examination (OSTE)

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Study about the oste
Study about the OSTE Engineering students

  • Used for formative assessment

  • Piloted with level 2 BEng students

  • Conducted with level 1 BEng students (1st + 2nd semesters)

  • Each student did the OSTE twice

  • Circulated feedback questionnaires for students + assessors

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Stages of the oste
Stages of the OSTE Engineering students

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Stage 1 development of oste stations
Stage 1: Development of OSTE stations Engineering students

  • Gathering a variety of ideas about essential electrical engineering skills:

    • Seminar within School

    • Call for ideas via Email

  • Identified appropriate range of exercises doable in 5 mins

  • 16 stations were short-listed

  • Developed instruction sheets & marking criteria for each station

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Stage 1 development of oste stations continued
Stage 1: Development of OSTE stations (continued) Engineering students

  • Types of stations:

    • Theoretical: pen & paper

    • Practical: assessor required

    • Computer-based: assessor required

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Stage 2 preparation for oste sessions
Stage 2: Preparation for OSTE sessions Engineering students

  • Development of PC-based clock with an integrated voice prompt

  • Briefing of students

  • Training of staff to act as assessors

  • Designing of student & assessor feedback questionnaires

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Stage 3 pilot of oste
Stage 3: Pilot of OSTE Engineering students

  • Group of 15 volunteer 2nd year BEng students

  • Resulted in changes in:

    • Tasks to be carried out

    • Marking sheet

    • Instruction sheet

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Example of a computer based station
Example of a computer-based station Engineering students

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Stage 4 first series of oste
Stage 4: First series of OSTE Engineering students

  • 1st year BEng students

  • During 1st semester (Dec 04)

  • Formative OSTE without feedback

  • To determine the level of students’ performance at the start of the course without preparing them too much for their 2nd OSTE

  • Feedback collected from students + assessors

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Stage 5 second series of oste
Stage 5: Second series of OSTE Engineering students

  • Same group of 1st year students

  • During 2nd semester (Mar 05)

  • Run as “mixed-mode” OSTE:

    (Alinier 2003)

    • Collection of data about students’ performance

    • Feedback to students by assessor

  • Feedback collected from students + assessors

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Results students perspective
Results: students’ perspective Engineering students

  • Out of 46 student feedback questionnaires

    • The OSTE was beneficial: 93.5%

    • The OSTE helped them developed confidence in using equipment: 73.9%

    • 52.2%: rated the OSTE as very good or good

      (5-point Likert Scale with 1=very good, mean=2.4 with 95% C.I. 1.4–3.4)

    • The OSTE should be used as a summative assessment tool: 28.3%

    • Number of times OSTE should be repeated per year: 2.9 (95% C.I. 1.4 - 4.4)

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Results assessors perspective
Results: assessors’ perspective Engineering students

  • Out of 33 assessor feedback questionnaires

    • The OSTE should be incorporated into the 1st year BEng curriculum: 93.1%

    • The OSTE helped students develop their confidence in using equipment they were exposed to: 75.8%

    • 90.6%: rated the OSTE as very good or good

      (5-point Likert Scale with 1=very good, mean=1.5 with 95% C.I. 0.8 – 2.2)

    • The OSTE should be used as a summative assessment tool: 75.9%

    • Number of times OSTE should be repeated per year: 2.2 (95% C.I. 1.4 - 3.0)

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Further analysis
Further analysis Engineering students

  • Identify key weaknesses among students

  • Any correlation between performance in OSTE and performance as per in-class assessment

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Oste results
OSTE results Engineering students

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Conclusions
Conclusions Engineering students

  • OSTE: perceived positively by students and assessors

  • Formative assessment 

  • Summative assessment ?

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


Thanks for your attention
Thanks for your attention Engineering students

Any questions?

[email protected]

[email protected]

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


References
References Engineering students

  • Harden R M, Stevenson M, Downie W W, Wilson G M, "Assessment of clinical competence using an objective structured clinical examination", Medical Education, Vol 1, 1975, 447-451.

  • Alinier G, "Nursing students’ and lecturers’ perspectives of OSCE incorporating simulation", Nurse Education Today, Vol 23, No 6, 2003, 419-426.

  • Alinier G, Hunt W B, Gordon R, "Determining the effect of realistic simulation in nurse education: Study design and initial results", Nurse Education in Practice, Vol 4, No 3, 2004, 200-207.

  • Ross M, Carroll G, Knight J, Chamberlain M, Fothergill-Boubonnais F, Linton J, "Using the OSCE to measure clinical skills performance in nursing", Journal of Advanced Nursing, Vol 13, 1988, 45-56.

  • Hulett S, Gilder N, "The application and development of the Objective Structured Practical Examination in evaluating physiotherapy students’ performance at the University of Cape Town", South African Journal of Physiotherapy, Vol 42, No 2, 1986, 40-43.

  • Marshall G, Harris P, "A study of the role of an objective structured clinical examination (OSCE) in assessing clinical competence in third year student radiographers", Radiography, Vol 6, 2000, 117-122.

  • Mossey P A, Newton J P, Stirrups D R, "Scope of the OSCE in the assessment of clinical skills in dentistry", British Dental Journal, Vol 190, No 6, 2001, 323-326.

  • Townsend A H, Mcllvenny S, Miller C J, Dunn E V, "The use of an objective structured clinical examination (OSCE) for formative and summative assessment in a general practice clinical attachment and its relationship to final medical school examination performance", Medical Education, Vol 35, 2001, 841-846.

  • Harden R M, "Twelve tips for organising an OSCE", Medical Teacher, Vol 12, No 3/4, 1990, 259-264.

  • Sloan D A, Donnelly M B, Schwarts R W, Strodel W E, “The Objective Structured Clinical Examination: The new gold standard for evaluating postgraduate clinical performance”, Annals of Surgery, Vol 222, No 6, 1995, 735-742.

N Alinier & G Alinier, ICEE 2005, July 25-29, Gliwice, Poland


ad