Academic Registry Question Time Annual programme review: how does EAP work this year?

1. Academic Registry Question TimeAnnual programme review: how does EAP work this year? Dr Jon Owen (Academic Registrar) Wednesday 9th September 2009 Thursday 10th September 2009 Tuesday 29th September 2009 Monday 12th October 2009

2. What are the major changes to APR? • the name, for distance: Evaluation of Academic Programme(s) • possible to “brigade” programmes together – no need for one report per programme; instead: report on cognate awards • a new format for statistics, using Excel drawn from the central student and academic information systems • a new template – no need to squeeze comments into boxes • no need to comment on every aspect of the statistics – just the “unusual” occurrences (report by exception) • focus on action plans • strengths and enhancement included • approach is one of evaluation supported by primary evidence • should be easier!

3. The cover sheet as document control • provided by the Academic Registry / Quality and Standards Unit • lists the programmes in the scope of the report and the name of the portfolio • it is still ok to have one programme per report • checklist of attachments: the primary evidence • summary of key performance indicators (see later) • signatures [cover sheet]

4. The primary evidence • statistics • external examiners’ reports • PSRB reports • National Student Survey feedback • student evaluations • moderators’ reports • employer feedback • programme committee minutes • recommendations from UWIC (re-)approvals • staff changes

5. The statistics • provided by the Academic Registry • five categories: • applications • admissions • progression, including reason for leaving (where applicable) • awards • career destinations • five years-worth of data, to show trends • tables and charts • same data set and conditions used for each presentation • presented by various properties [example statistics]

6. The statistics • properties: • gender • age • declared disability • ethnicity • domicile • type of entry qualification • socio-economic classification • “home” postcode (not yet available) • Welsh language ability • last three are not provided to overseas partners • available as a series of snapshots at any time during the academic year using standard reports

7. The action plan • refer to the primary evidence • rolls from one year to the next • unique identifier for each action [in the form 08-09.1] • what, where, how, when, who: • description of issue • source • description of action • when the action will be carried out • whose responsibility is the action • evaluation of effectiveness (likely to be in the next report) • explicit consent needed to assign “when” and “who” • assign a “traffic light” to each action

8. The action plan: an example • unique id: 08-09.1 • description: in module CRA002, Accounting for Engineers, students performed significantly worse than in other modules • source: external examiner report; module leader feedback • description of action: change the assessment method from a timed 3-hour exam to two items of coursework; it is inappropriate to test the preparation of the required documents within a specific time limit • when: with immediate effect • who: module leader • severity: • evaluation of effectiveness: [next year] [example]

9. The evaluation • Abstract • Background • Reflections on the key performance indicators • recruitment and admissions • student progression • award outcomes • career destinations • external examiner reports • student opinion • PSRBs (if applicable) • Personal development planning • What worked well • What needs to be improved • Good practice

10. The traffic lights • four possibilities: • red: serious issue requiring immediate attention • amber: major issue addressed in the action plan • amber/green: minor issue addressed in the action plan • green: no issues contributing to this performance indicator (so, by definition, does not appear in the action plan) • apply to the key performance indicators • be honest! • link to section 3 of the report • note that “staff opinion” does not have its own section in the evaluations: the whole report is “staff opinion”

11. The cover sheet as an at-a-glance summary Summary of key performance indicators [cover sheet]

12. Deadlines

13. What happens next? • in future, the AQSB may identify a “theme” for all authors to comment on (but has chosen not to do so this year) • Word template available on the Web for the body of the report • individual cover sheets will be distributed to the DLTs • statistics will be uploaded to the Web (self-service) • already providing requirements for the new student and academic information system • LTDU “enhancement day” • to come: cohort analysis rather than “year snapshot” • consistent with risk management: if provision is evaluated honestly and has few problems, then EAP for a given portfolio may not be required for a particular year