Designing Scoring Rubrics

1. Designing Scoring Rubrics

2. What is a Rubric? • Guidelines by which a product is judged • Explain the standards for acceptable performance or work • Points out what is significant or important in learning • Criteria for consistent evaluation • Guide students towards improved performance • Provide specific feedback to students on how to improve performance • Without a standard of evaluation, assessments are no more than instructional tasks

3. Criteria for Rubrics • Descriptors, traits, characteristics are terms used to indicate the elements contained in rubrics. • Reflect the elements of good performance and become the source for the criteria used to judge the performance. • Three types of criteria to determine grades: • Product, Process, Progress

4. Product Criteria • What students produce (pamphlet, brochure, play a game, create a game, perform a routine). • Performance and projects are examples of products. • Rubric contains critical elements necessary for required content. • Stating performance outcomes in terms of products says nothing about application or correct form. • These are process criteria.

5. Process Criteria • Critical elements necessary for correct performance. • Outlines the processes that students use to learn. • In Physical Education process criteria refers to the quality of physical performance. • Self-check, peer evaluation, teacher evaluation. • Evaluating process says nothing about how much a student has improved. • These are progress criteria.

6. Progress Criteria • Determine how much a student has improved or progressed. • Determines the mastery of the critical elements of performance. • In order to measure progress teachers must administer assessments more than once.

7. Types of Rubrics • The type of rubric chosen for assessment depends on the task being evaluated and the needs of the assessor. • Analytic and holistic rubrics are very complex and require time to generate. • Checklists are the easiest to create. • Point System rubrics are easy to convert to grades.

8. Checklists • Characteristics or behaviours that are basically scored as Yes/No ratings. • A list of characteristics regarding the performance or product is written and the scorer decides whether the trait is present or not. • Evaluate critical elements or process criteria of motor skills where quality is not a factor. • Journals or portfolios can be assessed to see if they have the presence of various points without making judgments about the quality of the response. • Checklists generally contain more traits or descriptors than usually found in other scoring guides. • This detail is helpful in guiding student learning.

9. Example: Checklist Rubric Yes No ____ ____ Plays within the rules of the game. ____ ____ Does not argue with others. ____ ____ Respect’s the other team’s efforts. ____ ____ Accepts the calls of officials.

10. Example: Self-Assessment ____ I play within the rules of the game. ____ I don’t argue with others. ____ I offer encouragement and support. ____ I accept the outcome of the game.

11. Point System Rubrics • Similar to checklists they award point for the various criteria on the list. • Judgment of quality is not required. • Provides students with feedback. • Points can be added up and converted to a grade. • Certain traits or characteristics can be weighted giving them greater emphasis.

12. Example: Scoring Rubric Fitness Portfolio ___ Fitness Evaluation (8 points) ___ Cardiovascular Assessment (4 points) ___ Abdominal Endurance (2 points) ___ Flexibility Assessment (2 points) ___ Fitness Plan (18 points) ___ Calculate target heart rate (2 points) ___ Needs analysis (4 points) ___ Workout Plan (12 points) ___ Warm-up and recording chart (3 pts + 1 point for chart) ___ Aerobic plan and chart (3 pts + 1 point for chart) ___ Strength plan and chart (3 pts + 1 point for chart)

13. Analytic Rubrics • Requires the scorer’s judgment to determine the degree of quality. • Evaluate the strength or weakness of a trait or element. • Students can easily see areas on which they must improve. • These rubrics take more time to score. • Words give a verbal indication of the degree of quality. • Always – Frequently – Sometimes – Never • 4 3 2 1

14. Quantitative Analytic Rubric • Game Play Assessment for Tennis • The Student: • Uses correct form on the forehand. • Places shots. • Moves into position quickly. • Returns to base position. • Calls shots correctly. • Knowledge of the rules is evident. • 1 2 3 4 • 1 2 3 4 • 1 2 3 4 • 1 2 3 4 • 1 2 3 4 • 1 2 3 4 Never Sometimes Frequently Always Useful for providing feedback to students and assessing their ability to utilize skills in an applied setting.

15. Qualitative Analytic Rubric • Rules for Tennis • Is unfamiliar with the rules. Depends on opponents or partner for instruction. Struggles with most questions. Is unsure of serving order and rotation. • Shows some evidence of knowing rules. Struggles with some questions. Serving order and rotation are correct. May have a few errors. • Shows evidence of usually knowing and applying rules. Can answer most questions when asked. Serving order and rotation are correct. • Shows evidence of thoroughly knowing and applying rules. Can answer any question when asked. Serving order and rotation are correct. Provide verbal descriptions of teacher expectations and are useful for providing formative feedback about several elements important to playing a game or performing well.

16. Holistic Rubrics –Useful for Summative Evaluations Proficient Level Consistently… Demonstrates the ability to select and successfully use the appropriate skills. Demonstrates good skill form. Moves to cover the appropriate space. Anticipates the offensive and defensive play and selects the appropriate response. Applies and follows the appropriate rules and scoring. Follows the rules of etiquette and fair play. Competent Level Demonstrates the ability to select and use the appropriate skills the majority of the time. Demonstrates good skill form a majority of the time. Attempts to move to cover the appropriate space a majority of the time. Selects the appropriate offensive and defensive strategy and the appropriate response most of the time. Consistently applies and follows the appropriate rules and scoring. Demonstrates good sporting behaviour consistently and follows game etiquette most of the time. Novice Level Inconsistently selects and uses appropriate skills. Frequently demonstrates incorrect skill form. Does not move to cover the appropriate space on the court. Little evidence of applying the appropriate strategy and response to a situation. Applies and follows appropriate rules and scoring. Inconsistent demonstration of rules of etiquette or good sporting behaviour.

17. Considerations to Address when Creating Rubrics • Validity – Does the rubric measure what it claims to measure. • Reliability – Does the assessment consistently produce the same results. • Transparency – Criteria are clear enough to students so that they can assess themselves and others with roughly the same reliability. • Subjectivity – The amount of judgment used to assign a score to a student’s performance.

18. Rubric Hints and Guidelines • Use samples of student work - samples show what is possible but should not limit the potential for excellence. • Share rubrics with students - criteria helps students understand teacher expectations. • Let students create some rubrics - students are more likely to understand important criteria. • Allow for multiple correct answers - sometimes the correct response is “it depends”. • Frequency is not the sole indicator of quality – too often when frequency is included in a rubric, the quality of the performance is not addressed.

19. Rubric Hints and Guidelines • Limit the scope of the assessment – complex rubrics are cumbersome and often capture non-critical elements. • Consider the levels of difficulty – give credit for attempting more difficult tasks. • Determine the number of levels to develop – the more levels, the smaller the difference between levels, and reliability decreases. • Adjust the rubric after, not during the assessment – changing the rules while the game is being played is not fair practice.