Promoting constructive alignment through programme specification and subject benchmarks

1. Promoting constructive alignment through programme specification and subject benchmarks Warren Houghton School of Engineering and Computer Science, University of Exeter

2. Plan. • Programme Specification • How we went about it in Engineering at Exeter • Discussion of process • Threshold standards • Defining differentiated assessment criteria at module level • Example and discussion • Helping students to manage their own learning • Levels of thinking about learning processes • A reflective framework for thinking about learning & teaching

3. Themes • Responsibility • Alignment • Reflection

4. Context of examples • University of Exeter – mid ranking “old” university – “research lead” • Department of Engineering • in School of Engineering and Computer Science • small general engineering department • 26 full time academic staff • approx 400 U/G students • Heavily constrained by PEI accreditation

5. Eng. & Management Eng. & Man Mechanical Electronic Civil Mechanical Electronic Civil 4-yr MEngaccredited 3-yrgeneralBSc 3-yr BEng accredited Context of examples Common first year

6. How we wrote Programme Specifications • Put Subject Benchmark Statement to one side ! • Wrote aims and ILOs for existing programmes • many iterations - emergent outcomes • Wrote aims and ILOs for existing modules • many iterations • drawing out what staff were already doing • Then, checked against Benchmark Statements etc. • Did not try to achieve one-to-one mapping

7. Why not use Benchmark Statements as “blueprints”? • What authority should we give the Benchmark Statements ? • What does the QAA say ? • Is there a “correct answer” ? • How can we obtain a set of required ILOs ? • From industry? • Do we have to take responsibility, with our own ideas?

8. William Perry’s positions: 1. Absolute right answers are provided by Authority. 2. Authority may make us find his absolute right answers ourselves. 3. Authority may not have found all the absolute right answers – yet . . . 4. Anyone has a right to his own opinion, but better keep Authority happy. 5. Everything is relative. 6. I may have to make some decisions for myself. 7. I must commit myself to a viewpoint. 8. I must take responsibility for what I commit to. 9. I am confident in my personal commitment, but I will keep an open mind.

9. What position do we take ? 1. Absolute right answers are provided by Authority. 2. Authority may make us find his absolute right answers ourselves. 3. Authority may not have found all the absolute right answers – yet . . . 4. Anyone has a right to his own opinion, but better keep Authority happy. 5. Everything is relative. 6. I may have to make some decisions for myself. 7. I must commit myself to a viewpoint. 8. I must take responsibility for what I commit to. 9. I am confident in my personal commitment, but I will keep an open mind.

10. What position are we encouraged to take ? 1. Absolute right answers are provided by Authority. 2. Authority may make us find his absolute right answers ourselves. 3. Authority may not have found all the absolute right answers – yet . . . 4. Anyone has a right to his own opinion, but better keep Authority happy. 5. Everything is relative. 6. I may have to make some decisions for myself. 7. I must commit myself to a viewpoint. 8. I must take responsibility for what I commit to. 9. I am confident in my personal commitment, but I will keep an open mind.

11. How should we use benchmarks? • NOT as some definitive “right answer” • We have to take responsibility for creating/recreating the curriculum • drawing on • our own experience • others’ experience - set out in benchmarks etc. • an iterative, reflective, process

12. Rewrite Programme Specification Implement changes Articulate currentaims, ILOs etc. Compare curriculum with Benchmark Statements etc. Contribute tonational discussion Read availableliterature. Benchmark Statements etc. Curriculum experienced by teachers and students Curriculum development Emergent outcomes

13. A limitation of benchmarks? • They are not explicitly multidimensional

14. A two-dimensional table for each assessment criteria heading MEng Assessment criteria Breadth and depth of programme BEng BSc 3rd 1st 2.2 2.1 Degree classification (performance)

15. MEng and BEng programmes : volumes in a space that is at least three-dimensional.

16. Dimensions of learning outcomes Order of thinking (e.g. Bloom’s hierarchy) “difficulty” of concepts Range of concepts

17. Simplistic “levels” view of a degree Order of thinking e.g. Bloom’s hierarchy Range of concepts

18. Allowing for other dimensions Order of thinking e.g. Bloom’s hierarchy Range or “difficulty” of concepts

19. Degree programme Prior learning and allowing for life before university ! Order of thinking e.g. Bloom’s hierarchy Range or “difficulty” of concepts

20. Plan . • Programme Specification • How we went about it in Engineering at Exeter • Discussion of process • Threshold standards • Defining differentiated assessment criteria at module level • Example and discussion • Helping students to manage their own learning • Levels of thinking about learning processes • A reflective framework for thinking about learning & teaching

21. Threshold standards • benchmarking implies . . . • all graduates will meet all threshold standards • we need to show how • we may have to change our assessment • QAA(2000) Engineering Benchmark Statement.

22. Threshold standards • Effective in (e.g.) the Royal Navy • Different in HE – why? • “Education and training are different” • Is certification realistic / useable ? • Too much / different for employers to read • PDP offers a solution to both problems

23. Example:setting and assessing threshold standards in core academic modulesin Engineering at Exeter

24. Traditional examination • 3 hr paper • Choose 5 out of 8 questions • Pass mark 40% • Pass provides evidence of 25% of ILOs tested • But which 25% ? • What can we build further learning on ?

25. define detailed ILOs/assessment criteria . . . • For all 1st and 2nd year engineering modules

26. Module specification – A/B structure

27. Assessment - examinations • Paper A: • Covering list A ILOs only • Typically, short straightforward questions • No choice • Expected mark >80% • Criteria referenced • Paper B • All ILOs, and some choice • Longer, more challenging questions with no “easy” parts • Expected average < 40%

28. Doesn’t this approach mean that we are blatantly teaching to the examination? Yes !

29. Development • Accepted because of PEI accreditation • evidence that students with different marks had achieved identifiably different learning outcomes • Originally developed as part of a scheme to give better guidance to students

30. Impact on staff • Staff find it hard to split ILOs this way BUT • asking for differentiated ILOs seems to work better than just asking for single level ILOs

31. Realism • Any test is demanding when the pass mark is 80% • If we are genuinely going to test all ILOs they must be achievable. • We have to be honest.

32. Deep vs. surface learning • Are we encouraging surface learning? • Other factors enable deep learning . . . • Consider structure of the learning (noun) • Hierarchy of concepts • What happens if students try to understand complex concepts when they haven’t grasped the components? • A/B approach makes deep learning possible

33. a problem • of success ?

34. Supporting students • A/B split originally introduced for student guidance • students asked to identify progress against ILOs on weekly basis • now whole of 1st year • ILOs are a prerequisite to PDP

35. Why is PDP important ? • It is about students becoming autonomous, independent, thinking for themselves • the real purpose of HE • It enables students to articulate what they can do

36. PDP: • is not a bolt on extra • it is an integral part of learning in HE • it must be addressed by all academic teaching staff

37. Academic staff • view PDP in qualitatively different ways and • many have difficulty with valuing PDP • Why?

38. Levels of thinking about teaching Biggs (1 to 3): Focus on: 1. what the student is 2. what the teacher does 3. what the student does plus? 4. how the student can manage what the student does (PDP)

39. Ways of thinking about Generic Graduate Attributes Barrie (1 to 4): 1. Necessary basic PRECURSOR skills but irrelevant as they are a prerequisite for university entry 2. Useful skills that COMPLEMENT or round out disciplinary learning 3. These are the abilities that let students TRANSLATE, make use of or apply disciplinary knowledge in the world 4. They are the abilities that infuse and ENABLE university learning and knowledge

40. 1. What student is 2. What teacher does 3. What student does 4. How student manages learning 1. PRECURSOR, irrelevant 2. Useful COMPLEMENT 3. TRANSLATE learning 4. ENABLE university learning Biggs - teaching(plus) Barrie - attributes (skills+)

41. Paradigm shifts Reflect Kolb Theorise Experience Plan

42. New understanding Paradigm shifts - double loop learning Paradigm shift

43. 1. What student is 2. What teacher does 3. What student does 4. How student manages learning 1. PRECURSOR, irrelevant 2. Useful COMPLEMENT 3. TRANSLATE learning 4. ENABLE university learning Biggs - teaching(plus) Barrie - attributes (skills+)

44. Learning to learn Teachers must : • have theories of learning • not just “bags of skills” (see Ramsden) STUDENTS need • study “skills” • AND • learning about learning • theories of learning • tools for metacognition / reflection / self management

45. Teacher as provider of tools for reflection on learning Student acquiring tools for reflection on learning Teacher as reflective practitioner Teacher as learning tutor Student discussing own learning Student engaged in PDP Teacher as subject tutor Student learning through subject related dialogue Teacher as creator of subject learning environment Student working within created learning environment A Reflective Framework

46. Teacher as creator of subject learning environment Student working within created learning environment The “created learning environment” must be designed so that students canmanage their own learning within it. It must offer: • real choices for students to make • resources to support different choices • information required to make choices (ILOs etc.) • assessment outcomes clearly linked to choices (i.e. aligned assessment) Student engaged in PDP

47. Teacher as provider of tools for reflection on learning Student learning about learning Teacher as reflective practitioner Teacher as learning tutor Student discussing own learning Student engaged in PDP Teacher as subject tutor Student learning through subject related dialogue Teacher as creator of subject learning environment Student working within created learning environment P.B.L. ?

48. Teacher as provider of tools for reflection on learning Student learning about learning Teacher as reflective practitioner Teacher as learning tutor Student discussing own learning Student engaged in PDP Teacher as subject tutor Student learning through subject related dialogue Teacher as creator of subject learning environment Student working within created learning environment A dialogue about learning is essential: PDP as a bolt on extra

49. PDP: • should be an integral part of the academic experience • requires teachers who are reflective practitioners • should have a profound impact on learning • can be used as a tool for curriculum development

50. Plan. • Programme Specification • How we did it in Engineering at Exeter • Discussion of process • Threshold standards • Defining differentiated assessment criteria at module level • Example and discussion • Helping students to manage their own learning • Levels of thinking about learning processes • A reflective framework for thinking about learning & teaching