Beyond Self and Peer Assessment David Boud

1. Beyond Self and Peer Assessment David Boud

2. Outline Part 1. What do we know about self and peer assessment? Part 2. How can we think differently about assessment: developing informed judgement? Part 3. How can we involve students as active agents in making judgements?

3. What is self assessment? the involvement of students in identifying standards and/or criteria to apply to their work and making judgements about the extent to which they have met these criteria and standards.

4. What does this imply? • Students may not be the only judges • Self assessment may be facilitated or left to chance • There may be different understandings of ‘involvement’ and ‘making judgments of learning” • There may be different practices involved • Self assessment doesn’t necessarily involve student involvement in summative assessment • Self assessment is not done in isolation from others or from standards

5. What do we know about self assessment? • Self assessment is a necessary skill for learning • within the course • after the course • Learners can be realistic in making self assessments • inexperienced learners and those new to an area tend tp overrate themselves • advanced learners are more realistic and can tend to underrate themselves • Context strongly influences ratings • incentives for students to overrate tend to work • Practice in making judgements improves self assessment • One-off uses of self assessment don’t have much impact

6. What else do we know? • Self assessment is not a method or technique, it simply represents who is the main agent in making judgements • Self assessment is best if not used in isolation from students considering other input • eg. of peers, of teachers, etc • Self assessment practices are extraordinarily varied and must be designed to fit the circumstances • of the subject • of the stage of development of the learner

7. Five qualitatively different ways of experiencing student self-assessment • Category A: as ensuring students’ behavioural compliance • Category B: as allowance for students’ contingent judgments of their knowledge • Category C: as providing feedback on students’ judgments of requisite standards in the program of study • Category D: as developing students’ judgments of their proficiency in the program of study • Category E: as sustaining students’ ability to self-assess beyond the program of study (Tan 2006)

8. Self assessment for formative and summative purposes • Self assessment for formative purposes can be fostered by learning tasks • In limited circumstances student marks can be used for summative purposes

9. Self/peer assessment may be used for grading when: • there is a high trust, high integrity learning environment • students are rewarded for high integrity marking • marks are moderated by staff so that deviations need to be justified • blind peer marking is used as a check • random staff marking is used as a check • students have had ample opportunity to practice and develop their skills • criteria have been sufficiently unambiguously defined to minimise misinterpretation of grade boundaries • effort is explicitly excluded as a criterion

10. Conditions in which self marking may be justified • when students are new to the knowledge domain and cannot yet recognise good work • when it is a preliminary stage to self assessment proper • when it is used in association with distancing devices to help students look afresh at their work • when rating scales used do not have connotations of what is socially desirable • when the sub-components of the task, not global marking are emphasised • when all scales and points on scales are explicit

11. Key features: self assessment • Active involvement in process, not following a recipe • Students involved in determining criteria, not just self-marking • Link to learning outcomes • May involve peers at some stage • Emphasis on informing judgment

12. Is self assessment flawed? Recent medical education literature points to the limitations of self assessment. How should we regard what they show? • Based upon meta analyses that typically demonstrate moderate correlations between self-judgement and those of teachers (~0.3-0.4) • Many of the empirical studies on which they are based are limited and were not designed to calibrate self assessment. • Use multiple sources of feedback to calibrate • Self assessment alone has, of course, considerable limitations for summative assessment purposes. • However, there is no choice but to persist with improving self assessment if learning is about developing judgement.

13. Peer assessment research 1 ‘Peer assessment seems adequately reliable and valid in a wide variety of applications, although virtually all of the current literature considers reliability of marks or grades rather than more detailed, formative assessment. Levels of acceptability to students are varied and do not seem to be a function of actual reliability. Students find peer assessment through tests, marks, or grades demanding but anxiety reducing. Learning gains in terms of test performance, skill performance, or subjective measures are frequently reported.’ (p. 268) Topping, K. (1998) Peer assessment between students in colleges and universities, Review of Educational Research, 68, 3, 249-276.

14. Peer assessment research 2 ‘Peer assessment and feedback of a more detailed, open-ended nature have been associated with improved confidence and better presentation and appraisal skills. The relatively high number and quality of studies of peer assessment of writing suggest outcomes at least as good as teacher assessment, and sometimes better. peer assessment of group and project work has been positive in terms of student perceptions. Similarly, peer assessment of professional skills shows adequate reliability but limited outcome data, often in participant perceptions. However, these again show outcomes at least equivalent to teacher assessment.’ (p. 268) Topping, K. (1998) Peer assessment between students in colleges and universities, Review of Educational Research, 68, 3, 249-276.

15. Using peer assessment • Use formatively in conjunction with self assessment • Peer feedback without summative elements can be used very widely • Peer assessment with a summative flavour must be used carefully otherwise it can inhibit the very learning it seeks to promote

16. Key features: peer assessment • As before for self assessment • Focus on peer assessment when communicating ideas to others is important • Use guidelines for giving and receiving feedback • Focus on qualitative peer feedback, downplay or eliminate ratings and grading

17. Offering feedback Be realistic Be specific Be sensitive to the person’s goals Be timely Be descriptive Be consciously non-judgmental Don’t compare Be diligent Be direct Be positive Be aware Receiving feedback Be explicit Be attentive Be aware Be silent Giving and receiving feedback

18. 3. Returning to assessment generally A. What to consider about assessment in higher education B. Contrasting models of educational assessment C. Thinking about developing students’ judgement D. Implications of viewing assessment as about informing judgement

19. A. What to consider in assessment in higher education • Assessment is about judgement. • Currently: judging learning outcomes against standards • Assessment must contribute to learning • for learning now • for future learning • Assessment is about both informing students’ judgements as well as making judgements on them • Summative assessment alone is to risky • Students must necessarily be involved in assessment • Assessment is a key influence in their formation and they are active subjects.

20. Scientific measurement model Practice derived from theory Knowledge is a ‘given’ for practical purposes Knowledge is ‘impersonal’ and context free Discipline-driven Deals with structured problems Judgemental model Practice and theory (loosely) symbiotic Knowledge is understood as provisional Knowledge is a human construct and reflects context Problem-driven Deals with unstructured problems (Hager and Butler 1996) B. Contrasting models of educational assessment

21. C. Thinking about developing judgement • Students must develop the capacity to make judgments about their own learning • Otherwise they cannot be effective learners now or in the future • We can never provide them with as much or as detailed feedback as students need. • Some kinds of feedback inhibit judgment through fostering dependency and compliance. • Capacity for self assessment is central to informing judgment • But simply adding self assessment activities is not sufficient. • Communities of judgment beyond ourselves need to be engaged with (peers, practitioners, professional bodies).

22. D. Assessment as informing student judgement implies • Always look to what the consequences of assessment are for learning • Focus on fostering reflexivity and self-regulation through every aspect of a course, not just assessment tasks • Recognise the variety of contexts in which learning occurs and is utilised—judgement is not independent of context • Stage opportunities for developing informed judgement throughout programs • Assessment must be integrated with learning and integrated within the program and over time

23. The problem of judgement • Judgement is more elusive than it appears • Making judgements are context specific and context dependent • There are intrinsic biases in making judgements • Judgement is always a subjective act, especially when the acts of people are judged • Getting to self-assessment is essential to judgement, but it is always flawed

24. Students as active agents in developing judgment • Student agency • Communities of judgment • Self assessment • Role of peers

25. Why involve students? • How can we justify not involving them? • But what does it mean to involve them? • Developing judgment is about more than acquiring knowledge and skills, it involves practice in discernment. • More opportunities for this are needed. • Students learn a lot through contributing to the learning of others • engaging with criteria, formulating ideas, taking account of the other • Students are a massively under-utilised resource at a time of resource constraints

26. Figure 1. Elements of informed judgement.

27. Some practices for involving students Don’t mistake practices for purposes! Not the practice but the purpose that counts. • Self assessment • Peer assessment • Hybrids Ultimately all assessment must be about informing the judgment of learners as it is only they who can learn.

28. Workshop task 1. Identify an assessment activity you wish to modify to enhance its contribution to learning in the longer term and developing student judgement. Make notes about what you will change. 2. In groups of three. Take turns in sharing your planned assessment activity and getting feedback. 3. Identify an issue that has arisen to bring back to the group.

29. Issues arising from the workshop task:

30. Practicalities • Common features for any assessment innovation • Choosing appropriate tasks and processes • Giving and receiving feedback

31. Common features in any assessment innovation • Assume students will be resistant, if they are not then they’re probably not behaving rationally • Never underestimate the importance of providing a compelling rationale and reiterating it in different ways • Act confidently. Listen to their concerns, but don’t change what you are doing unless they suggest a better way of doing it. • Be much more explicit than you imagine must be necessary, give full guidelines/deadlines, etc. in writing • Reassure students that cheating/ collusion will be detected if grading is involved • Be prepared to discuss tangible benefits to them

32. B. Choosing appropriate tasks and processes • ask learners to make judgments on matters on which it is reasonable for them to do so • use cues for success which are embedded in the content as much as possible • choose situations in which there are external sources of judgment which can be drawn upon or, multiple sources of evidence are available • avoid situations in which criteria for success are matters of opinion or taste • avoid incentives for mis-assessment • choose specific and concrete rather than the global and abstract task • limit the number of criteria which must be considered simultaneously at first • develop detailed guidelines on how the process is to be undertaken

33. Assessment is not enough • We can’t consider assessment separately from teaching and learning processes. All are about informing judgment. • Alignment between and integration of learning activities is needed • Choosing assessment practices chooses what students will learn

34. The new agenda • Not just constructive alignment, but alignment of assessment now with long term learning goals • Breakdown the binary between assessment and pedagogy • Revisit assessment from the perspective of lifelong learning • Revisit pedagogy from the perspective of lifelong assessment

35. References • Boud, D. (1995). Enhancing Learning through Self Assessment. London: Kogan Page. • Boud, D. (2000). Sustainable assessment: rethinking assessment for the learning society. Studies in Continuing Education, 22, 2, 151-167. • Boud, D. and Falchikov, N. (2006). Aligning assessment with long term learning, Assessment and Evaluation in Higher Education, 31, 4, 399-413. • Boud, D. and Falchikov, N. (Eds.) (2007) Rethinking Assessment in Higher Education: Learning for the Longer Term. London: Routledge. • Falchikov, N. (2005). Improving Assessment through Student Involvement. London: Routledge. • Gibbs, G. (2006). How assessment frames student learning. In Clegg, K. and Bryan, C. (Eds.) Innovative Assessment in Higher Education. London: Routledge.