Research Assessment Exercise 2005 (RAE2005) University of Helsinki Arto Mustajoki

1. Research Assessment Exercise 2005 (RAE2005) University of Helsinki Arto Mustajoki Based on the material of Vice-Rector Marja Makarow

2. Assessment of university research • University of Helsinki (UH) • University Senate’s decision • RAE of UH every 6th year • 1st time in 1999 (first in Finland) • 2nd time in 2005 • University of Tampere 2004 • University of Jyväskylä 2005 • Concept different from UK RAE • Carried out by UK Council, every 8th year • Peer review, no site visits • Important financial consequences

3. Why a RAE at the UH? • Current challenges for universities in Finland • Profiling • Strategic spearheads of research • International competitiveness • Productivity progam • Financial constraints • How to best meet the challenges • Knowledge of strengths, weaknesses and potential • External evaluation yields • solid objective data on quality of research • recommendations for the future

4. What was evaluated? • Quality of research of departments • Grade 1-7 (7 is best) • Verbal arguments • Concepts of institutes, research networks and stations • Only verbal arguments • No grades • Interaction with society • Only verbal arguments • No grades

5. What was evaluated?1 Quality of research • Quality of research compared to that of SIMILAR EUROPEAN INSTITUTIONS • Expressed with grades from 1 to 7 • Substantiated with verbal comments • NO comparison to results of UH REA 1999 • NO comparison between UH departments

6. Target of assessment • All research performed in UH during 1999-2004 • Units of assessment • Faculty departments: 70 in 11 faculties • Research institutes independent of faculties: 5 • Individual researchers or research groups were NOT evaluated

7. What was evaluated?2 Concepts • Verbal evaluation only of • Research (field) stations: 5 • Research networks: 5 • Independent institutes: 5 • Verbal evaluation of quality of research: • Medical research programmes: 6

8. What was evaluated?3 The third task – interaction with society • Pilot project • Legal obligation of universities since Aug 2005 • Documentation of accomplishments not systematic • For example • Expert tasks, popularization of science, text books, clinical and commercial application of research data • Units of assessment chose what to report • Evaluators commented verbally • The data serves to • Clarify how new knowledge has been tranferred to be used in society • Develop and document activities

9. How was the assessment carried out? • Peer review • Documents on acitivities of 94 units of assessment covering 1999-2004 via Evaluation Office to evaluators • Desk work at home • Preliminary drafting of Evaluation Report • Each panel one working week in Helsinki • General info on Finnish science policy, university system, UH • Site visits to premises, infrastructure • Interviews, researchers, PhD students, post-docs • Writing of Evaluation Report before leaving • Finalizing of the Report by Panel Chair • Editing the Report in Evaluation Office

10. Who were the evaluators? • 148 mostly international scientific experts, in 21 panels • From 21 countries • 83% from Europe • 9% from Finland • 12% were there in 1999 • 18% from LERU universities • 30% of panelists and chairs were women • Panelists were chosen from suggestions obtained from • Scientific Council of UH • Chairs of national Research Councils • Rectors of LERU universities • Intl top scientists

11. Research active staff (RAS) in 1999-2004 • Researchers, post-docs and PhD students • 4,000 annual work years • Results of altogether 24,000 researchers’ work years was evaluated • Number of RAS per unit of assessment: 3 - 640 • Humanities and social sciences: 3-76 • Natural science: 14-154 • Medicine 24-640 • Agriculture, Forestry, Bioscience, Pharmacy 7-340

12. Publications in 1999-2004 • Only publications in the official data base of UH were encluded in the assessment • Altogether 60,000 publications • 21,000 peer reviewed publications • 22,000 other publications • 2,400 monographies • 2,000 PhD theses • 10,000 popularized publications • 600 text books

13. Competitive funding fetched by the researchers in 1999-2004 (kiloEUR) • Research Councils (Academy of Finland) 220.000 • Ntl Technology Agency (Tekes) 61.000 • Ministries 127.000 • Ntl foundations 37.000 • Intl foundations 27.000 • EU FWPs 49.000 • TOTAL 521 M€

14. Results1 Grades of the quality of research Average 1999 4,66 Average 2005 5,8

15. Results3 Summary • From the 75 units 66 were evaluated also in 1999. • From them • 29 (44%) improved their grade • 31 (47%) got the same grade • 6 (9%) got a lower grade • 20 units (27%) got the best grade 7 • 6 units improved from 4 to 6 • 1 unit improved from 3 to 6

16. Criteria of the grades • 7: >50% of submitted works are at high intl level and all others are at good intl level • 6: >33% at high intl level and many others at good intl level • 5: >50% at least at good intl level and others at fair intl level • 4: >33% at good intl level and many others at fair intl level • 3: >50% at least at fair intl level • 2: >50% at fair intl level • 1: none at fair intl level

17. Quality of research increased - why? • Performance of individual researchers and teams • Structural development at university level • Concentration of activities to 4 campuses • Fusion of departments (115>75) • Evaluation culture adopted • Recommendations of RAE1999 implemented • Background • Sufficient national resources for research • Intelligent national science policy

18. Principles of financial consequences of results • University Senate’s decision before publication of results • The best units and faculties are rewarded • The resources will come from the university’s private funds, not state budget • No unit or faculty looses resources due to poor performance

19. Rewards to the best units of assessment • Units of assessment • 1.600 €/RAS/year will be awardes in 2007-2012 to units which obtained grade 7 • 1.600 €/RAS/year in 2007-2009 to units which improved to grade 6 from grade 3 or 4 • 27 units to be rewarded annually with • 30.000 - 288.000 €

20. Rewards to the beast faculties • Faculties, rewarded during 2007-2009 • 6 faculties with the best average grade • Amount of reward relative to number of RAS after deduction of the RAS of units to be awarded directly • 6 faculties to be rewarded annually with • 32.000 – 309.000 €

21. Total investment to quality research in 2007-2012 • 12 M€ to units of assessment • 3 M€ to faculties • Total investment 15 M€

22. Evaluators’ comments to leadership • Structures which best support quality research in universities • Independent institutes • Research programmes • Collaboration and strategic alliences • Infrastructure and its sharing • Proactive recruitment of researchers • Funding of research • Allocation of time for research • Researchers’ careers • Leadership

23. Governance of RAE2005Director of RAE2005 Vice-Rector for ResearchProf. Marja Makarow • Steering committee (Chair prof. M. Makarow) • Prof. A. Mustajoki • Director U. Mansikkamäki • Mr. H. Kallasvaara • Panels and panelists: • Steering committee >ReasearchCouncil of UH • ToR for evaluators & Guidance for units of assessment: • Evaluation Office (K. Haila & R. Holm) > Steering Committee > Reasearch Council of UH • Principles of financial consequences • Research Council of UH • Decision by University Senate

24. Publication of results • Duration of RAE procedure from May 2004 to March 2006 • Site visits of panels in Helsinki • May-June 2005: panels 1-4 • September-November 2005: panels 5-21 • Publication of results on the web on March 1, 2006 • Summary Report (also available in printed fomat) • Individual Evaluation Reports • www.helsinki.fi/research2005