Developing an effective assessment strategy

1. Developing an effective assessment strategy Peter Hartley, Professor of Education Development University of Bradford p.hartley@bradford.ac.uk

2. This session • introduction • what’s the problem? • introduce current projects and initial outcomes: the PASS and TESTA projects • identify important issues, suggestions and implications

3. Introduction • myself • good and bad news!

4. Introduction • myself • good and bad news! • The bad news – the jury is out! • The good news – growing evidence and examples from new projects and initiatives with the opportunity to build on and extend others’ experience

5. What’s the problem? #1 • Consider the issues buried in the title of this paper • the implications of ‘developing’ • effective ‘for whom’? • clarifying the purposes of ‘assessment’ • the notion of ‘strategy’

6. What’s the problem? #2 • See the PASS Issues Paper • Please comment/feedback and use • Would highlight: • Assessment ‘drives and channels’ • What/why are we measuring: the ‘slowly learnt’ • Limitations of grading systems(e.g. marks are not numbers) • Implications for course structures/regulations

7. Introducing the projectsPASS project #1 • NTFS group project over 3 years • One year of development and investigation and two years of implementation • Consortium • Led by Bradford • 2 CETLs – ASKE and AfL • Plus Exeter, Plymouth and Leeds Met.

8. PASS project #2 • Outcomes and outputs • approaches to PBA • ‘choice and consequence’ guides • workshop and resources for local implementation • case studies from different disciplines

9. TESTA project, #1 • NTFS group project with 4 partner universities • ‘aims to improve the quality of student learning through addressing programme-level assessment.’

10. TESTA project # 2 • starting from audit of current practice, so far looked at nine programmes • surveyed students using focus groups and AEQ – Assessment Experience Questionnaire – Graham Gibbs et al • also using tool to identify programme level ‘assessment environments’ (Gibbs)

11. Issues to disentangle include: • Defining assessment • Assessment environments and their impact • Defining ‘programme-based’ assessment • Student perceptions and expectations • The need for a strategic approach • Grading and credit

12. Defining assessment: a challenge • program outcomes “need to be assessed in complex, multidimensional student performances” • “Multidimensional performance entails the whole dynamic nexus of the individual’s intentions, thoughts, feelings, and construals in a dynamic line of action and his or her entanglement in an evolving situation and its broader context. Such a context may be within or across work, family, civic, or other settings.” • (Rogers, Mentkowski, & Reisetter Hart, 2006, p. 498).

13. Assessment environment and impact • Interim findings from TESTA • variety of assessments can cause problems • Issues over understanding assessment criteria, marker variation, and feedback • variation across programmes • QA ‘myths and traditions’ can get in the way

14. Starting to define PBA

15. Typical student perceptions and concerns (based on PASS) • perceptions of ‘the course’ are variable. • assessment experienced as ‘fragmented’. • anxieties re move to more integrated assessment – perceived risk in terms of performance. • Concerns about feedback and timing.

16. The need for strategy • An example finding from Gibbs • ‘greater explicitness of goals and standards was not associated with students experiencing the goals and standards to be clearer’ • And what did make a difference?

17. The need for strategy • An example finding from Gibbs • ‘greater explicitness of goals and standards was not associated with students experiencing the goals and standards to be clearer’ • And what did make a difference? • Formative-only assessment • More oral feedback • Students ‘came to understand standards through many cycles of practice and feedback’

18. An example: Peninsula Medical School • NB Case study forthcoming from PASS • Includes: • four assessment modules that run through the 5 year undergraduate medical programme and are not linked directly to specific areas of teaching • focus on high-quality learning (Mattick and Knight, 2007)

19. Issues re grading and credit • Teaching/assessment links • The assessment/credit link • Credit cf accreditation • Threshold cf grades in performance

20. And back to ‘effective assessment strategy’

21. And back to ‘effective assessment strategy’ • Will it explain to staff, students and external agencies: • How the course/programme assesses the main outcomes? • How assessment and teaching are linked? • How assessment both supports ‘high-quality learning’ and develops it over the course?

22. And finally … • Other initiatives and events, e.g. • EARLI conference later this year • Growth of work on feedback (e.g. use of audio as in ASEL)

23. And absolutely finally … Contacts for PASS • The project website • Project Director: Peter Hartleyp.hartley@bradford.ac.uk • Project Manager: Ruth Whitfieldr.whitfield@bradford.ac.uk