RAE Briefing by Keith van Rijsbergen at CPHC on 19th April, 2007

1. RAE BriefingbyKeith van RijsbergenatCPHCon19th April, 2007

2. RAE 2001/2008 • Two tier panel structure • Quality profiles • Outputs, not individual researchers are rated. • Fairness continues

3. outputs/individuals An underpinning principle is that sub-panels should assess each submission in the round: they will not make collective judgements about the contributions of individual researchers, but about a range of indicators relating to the unit, research group or department that is put forward for assessment (Generic; §21)

4. Size (2001) No. of submissions cross-refs • Pure Maths 47 3 • Applied Maths 58 3 • Stats 46 2 • CS 80 6

5. Timeline • June 2007: Consider survey of intentions... • January 2008: Allocate reading responsibilities • March 2008: Identify cross-refs & ext. adv • May 2008: Audit cross-ref advice • June/July 2008: First assessment meeting • September 2008: Continue assessment • Oct/Nov 2008: Final assessment meeting • November 2008: Submit profiles to RAE • December 2008: Results Published

6. CAVEATS • The published document RAE 01/2006 is the authoritative version. • The content was arrived at by collective discussion and debate. • The document contains compromises to achieve some compatibility across subpanels in F and to satisfy the lawyers in HEFCE+

7. Pure Mathematics (20) Applied Mathematics (21) Stats and OR (22) CS and Informatics* Ken Brown Tim Pedley Bernard Silverman Keith van Rijsbergen Main Panel F(Nigel Hitchin) * To be referred to as CS henceforth

8. International Members

9. Who is on our sub-panel?

10. Quality Levels

11. Excellence In assessing excellence, the subpanels will look for originality, innovation, significance, depth, rigour, influence on the discipline and wider fields, and relevance to users.(Main §15)

12. CS: 4* To be awarded a 4* quality level a research output must exhibit high levels of originality, innovation and depth, and must have had, or in the view of the sub-panel be likely to have, a significant impact on the development of its field. (CS: §66)

13. Generic:Section 20 These descriptive accounts should be read alongside, but do not replace, the standard definitions

14. Outputs All forms.....including, but not necessarily limited to: books, chapters in books, articles in journals, conference contributions; creative media and multimedia; standards documents; patents, product or process specifications, items of software, software manuals; technical reports, including consultancy reports or independent evaluations. All forms will be given equal consideration. (CS: §12)

15. Cross referrals &External advice Advice from other subpanels will be sought and given on the basis of the assessment criteria for the UOA to which the work was orginally submitted (Generic: §54) The advice of external advisers will be used to inform the subpanels’ assessment. (Main: §27)

16. ‘Reading’ As far as possible, the sub-panel will examine all research outputs, and expects to examine in detail at least 25% of the outputs in each submission. The subpanel will use its professional judgement to select a subset of outputs to examine in detail (CS: §17)

17. Research Environment • Infrastructure, facilities and administrative support for research • Arrangements for developing and supporting staff in research • Cumulative impact of research • Industrial collaboration, relationship with research users, • contribution to public awareness and understanding

18. Research Environment • Academic collaboration, national and international, within discipline and interdisciplinary • Research degrees awarded • Research income: funding strategy, amount received and sustainability • Credibility, vitality and sustainability of research organisation (CS: §27)

19. Esteem Indicators • awards, fellowships of learned societies, prizes, honours and named lectures • personal research awards and fellowships • keynote and plenary addresses at conferences • etc etc, see CS: §43

20. Citations • No panel will use journal impact factors as a proxy measure for assessing quality (Generic: §32) • They will not use a formal ranked list of outlets, nor impact factors, nor will they use citation indices in a formulaic way (Main §15)(CS: §19) • But.....In reaching these judgements the assessor may consider: citations of the output, relative to others of similar age in the same field (CS: §63)

21. Quality Profile

22. The overall quality profile 4* 3* 2* 1* u/c 15 25 30 20 10 is comprised of the aggregate of the weighted profiles produced for outputs, research environment and esteem indicators.(Annex 1) Overall Quality Profile Quality Level % of Research Activity Research Environment Esteem Indicators Research Outputs 70% 20% 10%

23. Some Do’s and Don’ts • Do read criteria and working methods carefully • Do submit early career researchers • Don’t provide info on detailed organisation and management structure of the department CS: §33

24. Take home message • Continue to produce high quality research, papers, etc • Continue to enhance research environment eg. grants, PhD’s, etc • Continue to attract awards, prizes, keynotes, etc • All the stuff we normally do.