1 / 17

Criteria for Good Assessment

Criteria for Good Assessment. John Norcini, Ph.D. Overview. Some background What is a ‘test’ or assessment? How does assessment affect learners? How are assessments changing? Who cares about assessment? Characteristics of good assessment How do they relate to purpose?

yen
Download Presentation

Criteria for Good Assessment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Criteria for Good Assessment John Norcini, Ph.D.

  2. Overview • Some background • What is a ‘test’ or assessment? • How does assessment affect learners? • How are assessments changing? • Who cares about assessment? • Characteristics of good assessment • How do they relate to purpose? • How do they relate to the stakeholders?

  3. What is a test or assessment? • Two definitions of ‘test’ • ‘To discover the worth of something by trial’ • Summative, evaluation • ‘To improve the quality of something by trial’ • Formative, assessment • For today, a ‘test’ or ‘assessment’ is any measurement process • Formative or summative • These are central to • Understanding the importance of assessment • Understanding the application of assessment • Identifying criteria for good assessment

  4. How does assessment affect learners? Assessment is a powerful tool in education Motivates students to learn Directs learning by signaling what it important Unintended effects when there is a mismatch between curriculum and assessment Ensures that learning has occurred Creates learning

  5. How are assessments changing? Past “That part of eternity with some small fraction of which we have a slight and regrettable acquaintance.” A Bierce Future “That period of time in which our affairs prosper, our friends are true, and our happiness is assured.” A Bierce Authenticity Active person/patient involvement More structured workplace training Life-long learning and quality improvement • Little realism • Little person/patient involvement • Lack of structure during workplace training • Episodic and infrequent continuing education

  6. Who cares about assessment? • Variety of stakeholders • Students • Key to learning and demonstration of competence • Teachers and individual/collective educational institutions • Accountability and self-assessment • Person/patients and healthcare systems • Accountability and efficiency • Regulatory bodies • Accountability and maintenance of competence

  7. Characteristics of good assessment • No single set of criteria apply equally to all assessment situations • Different purposes may lead to different values for the criteria • Different stakeholders and different mixes of stakeholders may value the criteria differently • Ottawa conference produced a set of criteria and how they are modified by purpose and stakeholders

  8. Characteristics • Validity or coherence • Is there a body of evidence that ‘hangs together’ and supports the use of assessment for a particular purpose • Is a property of the inferences drawn from a test, not the test itself • Is a matter of degree • Requires the ongoing collection of data “In seeking absolute truth we aim at the unattainable, and must be content with finding broken portions.” William Osler, 1889

  9. Characteristics • Reliability or reproducibility • Will scores be the same if examinees are reassessed? • History of reliability • Test-retest reliability • Alternate form reliability • Split-half reliability • Reliability index “Arithmetic…is a bad master, but it is a good servant.” Anonymous Lancet, 1921

  10. Characteristics • Equivalence • Different versions of an assessment yield equivalent scores or decisions • A challenge for assessment in the workplace • Catalytic effect • The assessment provides results and feedback in a fashion that enhances learning • A requirement for formative assessment

  11. Characteristics • Educational effect • Theassessment motivates those who take it to prepare in a fashion that has educationalbenefit • How do students prepare for the test? • Acceptability • Stakeholders find the assessment process and results to be credible • Feasibility • The test is practical, realistic, and sensible, given the circumstances and context

  12. Characteristics and Purpose

  13. Characteristics and Stakeholders

  14. Characteristics and Stakeholders

  15. Characteristics and Stakeholders

  16. Characteristics and Stakeholders

  17. Summary • Characteristics of good assessment • Validity, reliability, equivalence, catalytic effect, educational effect, acceptability, feasibility • Influenced by the purpose • Formative and summative • Influenced by the stakeholders • Learners, teachers-institutions, patients-healthcare system, regulators

More Related