Rubrics: Using Performance Criteria to Improve Student Achievement

1. Rubrics: Using Performance Criteria to Improve Student Achievement

2. Types of Assessment Methods • Selected-response • Require students to select a response from a provided list or supply a brief answer. • Examples: multiple choice, true/false, matching, or essay tests. NCATE Orientation Fall 2007

3. Constructed-response • Require students to construct a tangible product or perform a demonstration to show what they know and can do. NCATE Orientation Fall 2007

4. Constructed-Response • Students must organize and use knowledge and skills to answer a question or complete a task, rather than recall and recognize. • Cannot be scored with an answer key or by machine. • Assessments require a performance criteria scoring tool. NCATE Orientation Fall 2007

5. Performance Criteria • Are guidelines to use to judge student responses, products, or performances. • Describe what criterion or dimension to look for in student work. • Used to judge constructed response assessments. NCATE Orientation Fall 2007

6. Explicitly defined criteria • A goal for creating performance criteria is to make an essentially subjective judgment process as clear, consistent, and defensible as possible. • An effective scoring tool to use are rubrics. NCATE Orientation Fall 2007

7. Rubrics • Scoring tools that define the critical criteria of the performance to be assessed. • Contain levels of achievement or performance. • Levels provide detailed explanation of the degree of mastery and a numerical score. NCATE Orientation Fall 2007

8. When used for assessment rubrics are: • fair, • consistent, • practical, and • unbiased. NCATE Orientation Fall 2007

9. Benefits of Performance Criteria for Faculty • Consistency in Scoring • Clearly defined criteria communicate the important elements of quality. • Increase the consistency of judgments across teachers. • Improves Instruction • Assist in clarifying instructional goals and student learning outcomes. NCATE Orientation Fall 2007

10. Benefits of Performance Criteria for Students • Creates a shared vocabulary. • Develops understanding of the important dimensions of quality of performance. • Creates clear learning goals. • Removes mystery of expectations. NCATE Orientation Fall 2007

11. What Rubrics are NOT • Checklists • Likert scales • Performance lists NCATE Orientation Fall 2007

12. Two Types of Rubrics • Holistic: • Combines all performance criteria together to get single score or rating. • Analytical: • Divides a product or performance into essential traits or elements so they can each be judged separately. • Used when on going assessment is integrated with instruction and feedback is needed to improve teaching and learning. NCATE Orientation Fall 2007

13. When to Use a Holistic Rubric • Judge a simple product or performance. • Get a quick snapshot. • Often scoring on one dimension. • Disadvantages: • No detailed analysis of strengths and weaknesses. • Do not provide detailed feedback to students. NCATE Orientation Fall 2007

14. NCATE Orientation Fall 2007

15. When to Use an Analytical Rubric • Criteria is broken down into critical elements, content/coverage, of performance. • Judging complex performances involving several elements. • Provides more feedback to students. • Scoring on multiple dimensions. • Disadvantages: • More time-consuming to create. • Need to take time for training if more than one instructor is using the rubric. NCATE Orientation Fall 2007

16. NCATE Orientation Fall 2007

17. Analytical Rubric Learning Goals Rubric TWS Standard: The teacher sets significant, challenging, varied and appropriate learning goals. NCATE Orientation Fall 2007

18. Analytical rubrics, set a performance standard on each critical element. NCATE Orientation Fall 2007

19. More on Critical Elements • The descriptors under each critical element are not meant as a checklist. • Content of a rubric defines what to look for in student performance. • Rubric content is similar to content standards. NCATE Orientation Fall 2007

20. Clarity and Detail • The more detail the better. • Teachers, students, and others should be able to interpret the statements and terms in the rubric the same way. • Use samples of student work to illustrate what is meant. • Use words that are specific and accurate. • Levels are defined with lots of descriptive detail. NCATE Orientation Fall 2007

21. Technical Quality • Rubrics must adequately measure the skills being assessed. • Consideration of fairness- • Will the rubric treat all students the same way? NCATE Orientation Fall 2007

22. Determining the number of levels • Consider the nature of the performance and the purpose for scoring. • What is the range of qualitatively different degrees of understanding, proficiency, or quality of the product? • How many score points will distinguish quality? NCATE Orientation Fall 2007

23. Suggested: An even number (4 or 6) of levels of performance on the scale. • When there is an odd number of levels, scores tend to move to the middle. • With an even number of levels, raters have to make a more precise judgment about a performance when its quality is not at the top or bottom of the scale. NCATE Orientation Fall 2007

24. Steps in Rubric Development • Develop the critical elements to be assessed. • Determine “traits” – important dimensions of performance. • Determine if rubric will be holistic or analytical. • Brainstorm: • critical elements. • Comments that relate to each critical element. • Indicate which comments relate to each element at the high, middle, and low levels. • Review comments and place into categories. NCATE Orientation Fall 2007

25. Continuously refine: • Take notes as rubric is used on critical elements that may be missing and scoring levels. • Keep an open mind. • Get feedback from students. NCATE Orientation Fall 2007

26. Collect samples of student work that illustrate the various critical elements and performance levels. • Samples can help to refine rubric. NCATE Orientation Fall 2007

27. Barbara Chesler Buckner Associate Provost for Assessment and Accreditation Coastal Carolina University 843-349-6441 bbuckner@coastal.edu NCATE Orientation Fall 2007