Using Scoring Guides…

Using Scoring Guides… Current perspectives on classroom assessment and student achievement L.B. Uveges November 27, 2007

Research • Research consistently shows that regular, high-quality CLASSROOM ASSESSMENT increases student achievement

The key to improvement is how students and teachers USE assessment information

“Weighing the pig, doesn’t mean he’ll gain weight”

Assessment tools that calculate solely how well student achievement measures up to the standards, however reliable, will not suffice. Assessment must serve as a vehicle for improving the quality of learning for every student. • - National Research Council, 2001

Crunching the numbers • Grades based on averaging have meaning only when averaging repeated measures of similar content. Teachers average marks on fractions, word problems, geometry, and addition with marks for attendance, homework, and notebooks- and call it mathematics. In mathematics, we teach that you cannot average apples, oranges and bananas, but we do it in our gradebooks!” (*Ken O’Connor)

Ideas about assessment have undergone important changes in recent years. In the new view, assessment and learning are 2 sides of the same coin…when students engage in assessments, they should learn from those assessments. • - National Research Council, 1999

Formative Assessment: • All those activities undertaken by teachers and by their students (that) provide information to be used as feedback to modify the teaching and learning activities in which they are engaged. • - Black & Wiliam, 1998

Communication We must constantly remind ourselves that the ultimate purpose of education is to have students become self-evaluating. If students graduate from our schools still dependent on others to tell them when they are adequate, good, or excellent, then we have missed the whole point of what education is about (Costa and Kallick 1992)

Grades are merely symbols; in order to provide real information, they should be seen as only a part- probably a very small part- of our communication system • (Ken O’Connor)

Formative Assessment: 3 Guiding Questions • Where are you trying to go? • Where are you now? • How can you get there?

The QUALITY of the feedback rather than its existence or absence is what determines its power • - Bangert-Dewns, Kulik, Kulik & Morgan, 1991; Sadler, 1989

With regard to feedback, research makes the case for the use of DESCRIPTIVE, CRITERION-BASED feedback as opposed to numerical scoring or letter grades without clear criteria. • -Butler & Neuman, 1995

Research shows that feedback that EMPHASIZES LEARNING GOALS leads to greater learning gains than feedback that emphasizes self-esteem • -Ames, 1992; Butler, 1998

When receiving feedback emphasizing self-esteem, high-performing students often attribute their performance to effort and low-performing students attribute their performance to lack of ability.

Student SELF-assessment is crucial for feedback to be used effectively. Students are the ones who must ultimately take action to bridge the gap between where they are and where they are heading • - Sadler, 1989

Effective learners operate best when they have insight into their own strengths and weaknesses and access to their own repertoires of strategies for learning. • -Brown, 1994

In a year-long teacher-researcher collaborative project, Rudd and Gladstone (1993) helped foster self-assessment skills through questionnaires, concept maps, and self-assessment maps. They report the following results:

Development of students’ abilities to plan and think through their goals and skills • Creation of student awareness of the importance of evaluating their own work • Development of students’ abilities to evaluate each others’ self-assessment and provide constructive criticism • Increase in students’ abilities to manage resources and time more effectively.

The process of engaging in self-assessment increases students’ COMMITMENT to achieving important goals. • - Covington, 1992

SEVEN strategies for using a scoring guide as a a teaching tool: • #1- Teach students the language of quality- the concepts behind strong performance. • How do your students already describe what a strong product or performance looks like? How does their prior knowledge relate to the elements of the scoring guide you will use? How can you get them to refine their vision of quality

What you can do: • ASK students to brainstorm characteristics of good quality work • Show samples of work (low and high quality) and ask them to expand their list of quality • Ask students if they’d like to see what teachers think. • Have them analyze how student-friendly versions of the scoring guide match to what they said

#2- Read (view) score, and discuss anonymous sample products or performances • Some strong and some weak; some representing problems they commonly experience, especially the problems that drive you nuts! • Ask students to use the rubric to “score” real samples of student work. Since there is no single correct score, only justifiable scores, ask students to justify their scores using wording from the rubric. Begin with a single trait. • Progress to multiple traits when students are proficient with single-trait scoring

#3 Let students use the scoring guide to practice and rehearse revising • Its not enough to merely ask students to judge work and justify their judgments. Students also need to understand how to revise work to make it better. Begin by choosing work that needs revision on a signle trait

What to do: • Ask students to brainstorm advice for the author on how to improve his or her work. Then ask students to revise the work using their own advice (in pairs) • Ask students to write a letter to the creator of the sample, suggesting what she/he could do to make the sample strong for the trait discussed • Ask students to work on a product or performance of their own that is currently in process, revising for the trait discussed

#4 Share examples of products or performances from life beyond school- both strong and weak • Have them analyze these samples for quality using the scoring guide

#5- Model creating the product or performance yourself • Show the messy underside- the true beginnings- how you think through decisions along the way. • Ask students to analyze YOUR work for quality and make suggestions on improvement. • Revise your work using their advice • Ask them to again review it for quality

#6 Encourage students to share what they know • People consolidate understanding when they practice describing and articulating criteria for quality

Ask them to: • Write self-reflections, letters to parents, papers describing the process they went through to create a product or performance. Use the language of the scoring guide • Revise the scoring guide for younger students, make bulleted lists of elements of quality, develop posters illustrating the traits, or write a description of quality as they now understand it I used to … but now I…) • Participate in conferences with parents and/or teachers to share their achievement.

#7 Design lessons and activities around the traits of the scoring guide • Reorganize what you already teach and find or design additional lessons

Rubrics

Assessment Rubric: • TARGET- focused on target (PI’s) • METHOD- Does it match the target? • CRITERIA- Is the purpose / goal clear and specific? • TIMEFRAME- Does it meet time allotted/allowed? • SAMPLING- Are there enough questions to get a valid idea of students achievement? • VALIDITY- Does it measure what I want to measure? • BIAS/DISTORTION- Are there any elements of either bias of distortion?

Data Skills Self Assessment • Review the self-checklist • Rank from 0-5 (low-high)

PDSA/COAU?? • Plan-Do-Study-Act • Collect-Organize-Analyze-Use • Collect the data needed • Organize it (notebooks, excel, content areas) • Analyze (data tables, charts, graphs, trangulate, disaggregate) • Use- make decisions about learning- set criteria for measurement, improve student achievement

How do we get students to take responsibility? • Student involved classroom assessment • Student-involved record keeping • Student involved communication (portfolios, conferences)

But what about OAT? • Tests often are so broad in their coverage that the info is too imprecise for teahers’ to use • The delay between testing and score reporting is weeks or months away by the time the scores return, they no longer reflect the achievement of the learners • Tests are usually administered once a year while teachers make instructional changes …daily!

Learning Teams • Critique your assessments • Assist in moving the assessment project toward useful, successful goals • Follow a preplanned agenda that is management and applicable to your work

Tools • Goal setting- • How to teach the students how to goal set • Examples of goals • Collaborative goal setting • Goal setting plan • Long term planning • DATA folders

Setting/Environment • Ground rules • Interviews/introductions • Capturing expectations • Issue Bin (Penalty Box) • Plus/Delta • Mission/Vision statements

Tools for what they know: • Brainstorming • Affinity Diagram • Evaluation criteria • Light-voting • Force-field analysis

Tools for thinking process: • Flowchart • Interrelationship diagram • Fishbone diagram • Tree diagram • Action planning

Tools for gathering data • Check sheets • Surveys (surveymonkey.com) • Pareto chart • Matrix • Scatter diagram • Run chart

Using Scoring Guides…