Preparing for QAA Academic Subject Review Norman Jackson LTSN Generic Centre

1. Preparing for QAA Academic Subject ReviewNorman Jackson LTSN Generic Centre • National infrastructure for improving T&L • LTSN and its relationship to QAA • What is QAA trying to do? • Overview of QAA policies and how they relate • Programme Specifications • Subject Benchmarking • Codes of Practice • Academic Review

2. National infrastructure for improving T&L 1992 2002 QAA HEQC ILT Capacity HEFCE regulation LTSN development research ESRC

3. LTSN is not an arm of QAA In working with the QAA agenda LTSN sees its role as * helping HE communities to understand the implications of QAA policies and make effective use of them * providing practical advice based on research and exemplification * promoting an evidenced-based approach to evaluating policy and its impact * influencing QAA thinking on the way policies are interpreted * encouraging HE to be self-critical in order to learn and improve.

4. LTSN values we believes that: there is value in subject communities trying to understand the nature of learning in their subject and describing this for the benefit of the whole community. there is value in teaching teams * being explicit about the learning they are intending to promote * explaining the methods they will use to enable students to learn * using information provided by the subject community to reflect self critically on their practice

5. What is QAA policy trying to do? HEI purposes and policies explicit intentions processes and outcomes external reference points

6. QAA policy framework INSTITUTION SUBJECT NATIONAL subject benchmarks programme specifications & progress files qualifications framework & codes of practice explicit intentions processes/outcomes external reference points Statutory/Prof. Body requirements

7. Academic Subject Review INSTITUTION SUBJECT NATIONAL subject benchmarks programme specifications & progress files qualifications framework & codes of practice explicit intentions academic subject review self-evaluation external reference points Statutory/Prof. Body requirements

8. QAA policies are promoting a consistent language for the description and assessment of learning • An outcome is simply a result or consequence of an action or process. The outcome from a learning process is therefore a learning outcome. • Learning outcomes are statements that predict • what learners will have gained as a result of learning

9. Example learning outcome statements at the end of the degree programme students will • Knowledge & understanding • have knowledge of the principal features of the English Legal System, • including general familiarity with its institutions and procedures • show they understand the theory of electronic devices,circuits & systems • Cognitive skills • be able to select and apply appropriate management techniques to • complex problems, analyse results, draw appropriate conclusions and • present results in an appropriate format • Subject practical skills • have mastered fundamental synthetic techniques in organic and • inorganic chemistry with practice in related quantitative analytical techniques • General transferable skills • have demonstrated that they can communicate succinctly orally • and in a variety of written formats

10. Explicit learning outcomes curriculum design & TLA methods assessment criteria intended learning learning process learning achieved

11. Explicit learning outcomes generic assessment criteria? programme specification learning achieved intended learning learning process subject benchmark performance criteria subject benchmark learning outcomes

12. Programme Specification Core information Educational aims Teaching, learning and assessment methods to enable student learning Intended learning * knowledge/understanding * skills and other attributes

13. Understanding how ouctcomes are enabled 1 2 3 4 5 6 7 8 etc curriculum building blocks A B C D E F G H TPA TPA P TPA P T=taught P=practised A=assessed TPA TPA TPA TPA TPA P TPA TPA P programme ouctomes TPA TPA TPA TPA PA TPA PA TPA TPA TPA TPA P P

14. HE Progress File PDP - A structured and supported processundertaken by an individual to reflect upon their learning,performance and / or achievement and to plan for their personal, educational and career development Transcript Personal Development Planning Personal Records goals enhance the capability of students to learn through reflection improve information about learning and achievement

15. PDP is an integrated and strategic process • containing a set of interconnected activities : • recording (thoughts, ideas, experiences, evidence of learning) • reviewing (making sense of it all) • evaluating (judging self and determining what needs • to be done to develop/improve/move on) • planning (how to achieve objectives or desired change) • doing (learning through the experience/learning more • or more effectively because of greater awareness) • making effectiveuse of knowledge & information about self

16. There is nothing new about PDP The process of reviewing (revising) what has been learnt in order to crystallise newly acquired knowledge and consolidate understanding and then identify and rectify gaps and deficiencies is central to effective student learning. PDP simply captures the intellectual capacities, skills and behaviours that underlie these processes and uses them to help people review and evaluate themselves in a structured way.

17. Possible contexts for PDP • Academic curriculum • academic modules/units • purpose designed modules/units • research projects and dissertation work • academic tutorials • work placement or work experience • study overseas • strategies that promote a commitment to CPD • Work-based / practice-based experience • Extra-curricula activities • student training initiatives • working as a student rep. or SU officer • part-time paid or voluntary work

18. General characteristics of benchmarking What is a benchmark? QAA benchmarking * reference point * criterion for measurement * mark of distinction (excellence) Benchmarking contains three ideas * searching for, creating and understanding the benchmarks * evaluating own practice, process  results with benchmarks * changing practice to emulate  or exceed benchmarks   X X

19. Subject benchmark statements are intended to embody * defining principles or essence of a subject * nature and extent of a subject – ‘subject territory and programmes included in territory’ * attributes that a graduate in the subject might be expected to display and demonstrate in terms of the subject knowledge and understanding : subject skills and other skills * the criteria that would be used to determine whether a graduate satisfied the ‘threshold’ standard for the award of an honours degree in the subject.

21. Benchmarking as a reflective tool benchmark learning outcomes curriculum & TLA methods programme outcomes benchmark performance criteria assessment criteria used within the curriculum generic assessment criteria ?

22. Possible uses of benchmark performance criteria 1. Checklist for assessment criteria in key final level units that permit a range of outcomes to be demonstrated 2. Checklist for assessment criteria in compulsory final level units 3. Checklist for assessment criteria for all final level modules 4. Checklist for assessment criteria in all curriculum units

23. Code of practice for the assurance of quality & standards • a reference point for HEIs and peer reviewers • Collaborative provision • Students with disabilities • External Examining • Academic appeals and student complaints • Assessment of students • Programme approval, monitoring and review • Postgraduate Research Programmes • Guidelines for Distance Learning • Careers advice and guidance • Work placements     most important for self-evaluation at programme level

24. HE minister Margaret Hodge June 15th THES ‘we want to ensure that we have high quality teaching in all our universities. We need to develop something that has the respect of all the players.’…. ‘Students care a lot about teaching standards.’ ‘the focus will be on the student, the removal of barriers to university entry and raising standards.’ our ambition - 50% under 30s to have access to HE

25. Blunkett called for a 40% reduction in visits The HEFCE/UUK position statement proposes the following visit pattern * external review of all departments who did not achieve the threshold performance level in Subject Quality Assessment. * external review of a sample of departments/ schools achieving an excellent rating (using the methodology before 1995) or who achieved a rating of 3x4 and 3x3pts in the six aspects using the 1995-2001 methodology.

26. All departments/schools will be expected to produce • * a self-evaluation describing the characteristics, • strengths and areas for development of the • programmes it offers within each subject area • This will not be published in order to encourage frankness. • * all departments/schools will be expected to publish • (ie make public) programme specifications for the • programmes they offer. • These will be an important source of public information.

27. Academic Subject Review differs from TQA 1. Underpinned by policy framework (PS, Subject Benchmarking, Codes, Qualification Framework) 2. Form of peer review process is negotiated 3. sampling rather than universal visiting 4. Main focus on intended and achieved student learning and how it is enabled 5. Generally, no observation of teaching 6. Judgements not overtly based on points Like TQA underpinned by self-evaluation

28. Self-Evaluation Document provides the framework for review • Aims of subject provider • Learning outcomes • Curricula & assessment • Quality of learning opportunities • Teaching and Learning • Student progression and achievement • Learning resources • Maintenance /enhancement of standards and quality • Appendices • Programme specifications • Information about curriculum structures • /collaborative arrangements

29. A sea change in the balance of accountability & development Academic Subject Review will now be a developmental and improvement-led process (self-evaluation in the light of external reference points coupled to changes made in response to self-evaluation). For many departments/schools professional and public accountability will be within the institution’s own QA procedures and through the provision of the self-evaluation to QAA reviewers. For a proportion of departments/schools there will also be an external accountability dimension to the process. The goal should be to achieve the first position for all departments/schools.