Educationalassessment
This presentation is the property of its rightful owner.
Sponsored Links
1 / 40

EducationalAssessment PowerPoint PPT Presentation


  • 53 Views
  • Uploaded on
  • Presentation posted in: General

EducationalAssessment. Classroom MCQs Clinical OSCE. Objectives. By the end of seminar learners will be able to: Define assessment Overview of assessment and educational assessment Explain attributes of assessment

Download Presentation

EducationalAssessment

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Educationalassessment

EducationalAssessment

Classroom

MCQs

Clinical

OSCE


Objectives

Objectives

By the end of seminar learners will be able to:

  • Define assessment

  • Overview of assessment and educational assessment

  • Explain attributes of assessment

  • Describe multiple choice questions (MCQs)

  • Discuss the steps of designing MCQs.

  • Identify advantages and disadvantages of MCQs

  • Define OSCE

  • Discuss the process of OSCE

  • Converse approaches/models of OSCE

  • Debate on OSCE

  • Discuss Ethical issues in OSCE

  • Know the views of students and faculty for OSCE


Definition of assessment

Definition of Assessment

  • Assessment comes from the old French word assessor, meaning to sit by as an assistant judge or guide.

  • The process of collecting information on student achievement and performance. Provides information for basis of sound decision making regarding teaching and learning.

    http://www.aac.ab.ca/final


Educationalassessment

Cont…

  • “The process of assessment is to gather, summarize, interpret, and use data to decide a direction of action”

    (Bastable 2003)

  • “Begins with educational values, works best when program has clear purposes…is ongoing and meets the responsibilities to students and public”

    (Billings 2003)


Educational assessment

Educational Assessment

  • “Is the process of documenting, usually in measureable term, knowledge, skills, attitudes, and beliefs”

  • “It focus on the individual learner, the learning community, the institution, or the educational system as a whole”

    http://en.wikipedia.org/wiki/Assessment


Overview of assessment

Overview of Assessment

what

why

Whom

Of What

Assessment

where

How

when


Assessment

Assessment

  • What? process of gathering…

  • Why?

    • Measure academic achievement

    • Identify areas for improvement

    • Predict future performance


Of what

Domains

Cognitive

(Knowledge)

Affective (Attitude)

Psychomotor (Skills)

Content

Facts and principles

Processes

Signs and symptoms

Clinical methods

Techniques

Management

Etc….

Of what?


Educationalassessment

How?

  • Written tests

    • Essay

    • True false

    • MCQs

    • MCIs

    • Viva/oral

    • Practical

    • Long case/short case

    • Osce

    • Projects


Educationalassessment

How?

Performance assessment in vivo

Undercover SPs, Video, Logs only in real world of

practice

Does

Performance assessment in vitro

OSCE, Standardized-patient based test

Shows how

(Clinical) Context based tests

MCQ, essay, oral

Knows how

Factual test: MCQs, Essay, Oral

Knows

Both areas tested by traditional written exams….

(Miller 1990)


Educationalassessment

When?


Where

Where?

  • Testing in situations similar to

    those in training and practice,

    enable to make justifiable

    decisions


Assessment types

Assessment Types


Attributes of assessment

Attributes of Assessment

Curricular attributesinclude

  • Program and course objectives

  • Critical learning experiences

  • Learning outcomes

    Faculty attributes comprise

  • Level of content knowledge

  • attitude about teaching

  • Instructional skills

  • Appropriate knowledge of teaching learning theories

    Students’ attributes consist of

  • Knowledge and skills

  • Cognitive abilities

  • Learning styles

  • Motivation to achievement (Billings 2005)


How to develop effective assessments

How to develop effective Assessments?

  • Should be RELIABLE and VALID

  • Reliability: A reliable assessment is one that provides consistent “scores”.

  • Validity: An assessment is valid when it accurately measures what is to measure.


Multiple choice questions mcqs

Consists of two

parts

Stem

Can be a question or an incomplete statement, scenario, problem

Several options/ distracters

But to choose the best response

Multiple Choice Questions. MCQs


Steps in designing mcqs

Steps in Designing MCQs

Stem

  • Clear single problem

  • Content should be common

  • Independent of all others in the test

  • Information in one stem should not give clue to another item

  • Scenarios incorporated carefully

  • Use of action verbs of Bloom taxonomy (1956) according to cognitive level of students

  • Knowledge, comprehension, application, analysis

  • (Billings, 2005)


Steps in designing mcqs1

Steps in Designing MCQs

Options

  • Grammatically consistent with stem to avoid giving clues to right one

  • Arrangement order, alphabetical, numerical

  • Having same length

  • Give only one best option, to which all faculty agree

  • Should not use all of above, none of above


Examples of mcqs

Examples of MCQs

  • Knowledge

    1. A patient’s father died of Huntington’s chorea. What are the chances that pt will have the disease?

  • 25%

  • 50%

  • 75%

  • 100%


Comprehension

Comprehension

2. A pt resists the nurse's attempts to get him out of bed. If this attempt is without pt’s approval, what legal charge could nurse face?

  • Assault

  • Battery

  • Negligence

  • Tort


Application

Application

3. On the 1st post-op day after an open reduction and internal fixation (ORIF) of the tibia with application of long leg cast, the pt begins complaining of severe pain uncontrolled by his scheduled pain med. What would be the priority nursing intervention?

  • Administer a narcotic bolus as ordered

  • Assess the neurovascular status of leg

  • Lower the leg to increase arterial flow

  • Raise the leg to decrease venous return


Analysis

Analysis

4. A 70 yrs old woman is admitted to the hospital with a diagnosis of dehydration. Vital signs are stable. Serum sodium level is 165 mmol/L. Which of the fluid replacement is most likely to be administered?

  • Hypertonic

  • Hypotonic

  • isotonic


What makes mcqs good

What makes MCQs “Good”?

  • Alignment: to content, learning outcomes/goals,

  • Importance: testing details is a waste of resources; concentrate on fundamental concepts

  • Clarity: the intent of the task and the meaning of the options must speak for themselves and be interpreted in the same way by all examinees

  • Item construction: well-constructed items can be answered successfully by examinees who have the knowledge or skill you are testing, and NOT by examinees who are lacking the skill or knowledge


Educationalassessment

Advantages

Large content in single test

Scored easily and objectively

Measure understanding at numerous cognitive levels in taxonomy category

Disadvantages

Time consuming in designing for faculty

More time needed by students to read and understand

Differentiate against creative, verbal student

Writing style affects scores

Difficult to write MCQs at synthesis and evaluation level

MCQs


Educationalassessment

Inspect

Palpate

Percuss

Auscultate

OSCE

CPR being performed on a mannequin used for training


Objective structured clinical examination

Objective Structured Clinical Examination

  • OSCE is a competency-based evaluation aimed at testing the psychomotor and affective domains with an inbuilt system for systematic feedback.

  • OSCE is a modern type of examination often used in health sciences (e.g. medicine, physical therapy, nursing, pharmacy) to test clinical skill performance and competence such as communication, clinical examination,


Educationalassessment

Cont…

medical procedures / prescription, exercise prescription, joint mobilization / manipulation techniques, and interpretation of results.

http://www.osceskills.com/

  • Also known as “Standardized Patient Exams SPEs, described as an acceptable and powerful instrument in clinical performance evaluation… pretended patients in an artificial environment designed to simulate actual clinical condition”

    (Borbasi & Koop 1994 as cited in Billing 2005)


Process of osce

Process of OSCE

  • It normally consists of several short (5-10 minute) stations, in which each is examined on a one-to-one basis with an impartial examiner and either real or simulated patients (actors).

  • Considered to be an improvement over traditional examination methods because the stations can be standardized enabling fairer peer comparison and complex procedures can be assessed without endangering patients health


Educationalassessment

Cont…

  • The other ways an OSCE is made objective is by having a detailed mark scheme and standard set of questions.

  • For example, a station concerning the demonstration to a simulated patient on how to use a Metered dose inhaler would award points for specific actions which are performed safely and accurately (e.g. candidate explains to patient the need for a seal around the mouthpiece, etc.)

http://www.osceskills.com/


Approaches models to osce

Approaches/Models to OSCE

Multi station

OSCE

AKU Med students

1st & 2nd year

OSPE

Scenario based OSCE

(sometimes called obj

structured clinical

assessment

(OSCA)

OSCE

3rd & 4th year

Top to Toe

Assessment

OSCE /simulated

Assessment with random

allocation

Some centers also use viva, or Q/A with

these models

(Rushforth 2007)


Is osce debatable

Is OSCE debatable?

OSCE stations can never be truly standardized and objective in the same way as a written exam.

  • It has been known for different patients / actors to afford more assistance, and for different marking criteria to be applied (literature shows marking criteria as pass/fail, checklist, likert scale, percentages, done/not done)

  • It provides a formative evaluation for both students and the educational institution.

    (Rentschler et al, 2007)


Educationalassessment

Cont…

  • It is a fast and well-organized evaluation method, allows rapid feedback to students about their clinical deficits (Billings 2005)

  • OSCE has been tested and found to be reliable, valid in various settings, provides a controlled clinical situation that is realistic and nonthreatening

  • # of stations 16 minimum, 37 max

    (as cited in Rushforth 2007)

  • Easy in controlled setting but difficult in real world

    Literature says that it should be mandatory for nursing students as well

    (Rentschler et al, 2007)


Ethical issues in osce

Ethical Issues in OSCE

  • Perceived inequity between students

  • Lack of honesty and professionalism of peers

    http://jme.bmj.com/cgi/content/abstract/32/12/734


Educationalassessment

Cont…

  • Discuss the OSCE cases with other students after the OSCE?violation of the honor code.

  • Why not children as simulated patients?

    http://www.oucom.ohiou.edu/AcademicAffairs/predoc/osce/index.htm


Views of students and faculty

Students

Overall satisfied

Stressful

Improve performance

Can practice on peers as well

Encouragement and motivation

Faculty

Overall satisfied

Students practice actively

Provide evaluation to students soon after exam

Views of students and faculty


References

References

Billings,D.M,Halstead,J.A. (2003).Teaching in nursing: A guide for faculty. 2nd ed.

Bastable,S.B.(2003).Nurse as educator:principles of

teaching and learning for nursing practice. 2nd ed.New York: Jones and Bartlett Publishers.

Hammond,L,D & Bransford, J.(eds) (2005). Preparing

teachers for a changing world:what teachers should learn and be able to do. USA. Jossey Bass. pp 275-326


Educationalassessment

Cont…

http://www.oucom.ohiou.edu/AcademicAffairs/predoc/osce/index.htm

Norman, G. (2002) The long case versus objective structured clinical examinations. BMJ324: 748-749

Objective Structured Clinical Examination. Retrieved Jan 29, 2009 from http://www.osceskills.com/

Parks,R.,Warren,P.M.,Boyd,K.M., Cameron,H.,Cumming, A., & Jones, J.L. (2006).The Objective Structured Clinical Examination and student collusion: marks do not tell the whole truth.Journal of Medical Ethics,32. 734-738


Educationalassessment

Cont…

Rentschler,D.D; Eaton,J; Cappiello,J; McNally,S.F; & McWilliam,P.(2007). Evaluation of undergraduate students using objective structured clinical examination. Journal of Nursing Education, 46 (3) 135-139

Rushforth, H.E. (2007). Objective structured clinical examination (OSCE): Review of literature and implications for nursing education. Nurse Education Today ,27, 481-490

Stiggins,R. J.(1999). Assessment, Student Confidence, and School Success. Phi Delta Kappan,81, (3) Retrieved Jan 23, 2009 from http://www.aac.ab.ca/final 2002.doc


Educationalassessment

Questions


Educationalassessment

THANK YOU


  • Login