Experiences from evaluation

1. Experiences from evaluation Petri Suuronen, Research Director Finnish Game and Fisheries Research Institute (FGFRI) International Evaluation of Water Research in Finland Säätytalo, 1 April 2008

2. Timetable for preparations was tight • First information of the evaluation received in September 2006. • Detailed request came 11 April 2007. • Dead line for the material was 14 May 2007. • The rather short time (1 month) allowed for collecting the evaluation material caused some difficulties in the institutes.  This may have affected the quality of the data collected.

3. Interpretations in the evaluation material • The Evaluation covered water research. • In research institutes there are various expert services not directly connected to research work. • Many scientists work both on research and expert activities. • The location of the borderline between research and other tasks is often fairly difficult to define (fuzzy). • It is possible that the man-months used for research are not presented in an exactly uniform manner (within and between institutes).

4. Imperfect data • The data request for the evaluation indicated that institutes may have shortcomings in their routine systems of recording various activities. • For instance, we noticed that we had no database available concerning the participation of our staff in radio and TV-programs. • Therefore, our data for the evaluation are imperfect at least in this respect.

5. Quality of publications – how to measure? • Senior scientists had to list their five key publications in the quality order. • The word “quality” was open to interpretations; it could mean: • relevancy of the study, • the quality of the journal (impact factor), or • the number of the citations to that paper. • It is obvious that the key publications are not listed in an exactly uniform manner.

6. Expertise of the Evaluation Panel • Water research is a wide field. • Fisheries research is a small part of it (game research even smaller). • It is almost inevitable that some fields are better represented in the evaluation panel (than some other fields). • It appeared that the panel, although relatively small, had a wide expertise in the water research, and good understanding even about fisheries research. • The discussion between the Panel and FGFRI representatives was very fruitful and constructive. • We certainly were not undervalued .

7. Relevancy of the research • Scientific quality is the cornerstone of the work of any research institution. • For the governmental research institutes also the relevancy of research is a fundamental factor. • Institutes work in a turbulent world where the research results are brutally criticised if they do not support the view of the stakeholders.    literally between the devil and the deep blue sea. • Strategic choices of the research (i.e., relevancy) would have been an important item to assess and evaluate. • The short time period did not allow in-depth discussions about the relevancy.

8. Research infrastructure scattered • The water research in Finland is scattered all over the country and there are plenty of players within the field. • Finland is a land of field research stations! • The Panel could have visited and familiarized with some of the aquatic field stations in various parts of Finland. • An evaluation of the optimal number and locations of research stations could have been useful? • Promotion of cooperation important.

9. Recommendations made by the Evaluation Panel • Despite of the small troubles described above, the International Evaluation has generated a highly useful outcome for the future development of the Finnish water research. • The Report includes 11 clear recommendations, which will be our guidelines for the coming development actions.

10. (a) Shift from mass-production of PhD’s • A structural weakness was observed in the Finnish education system.  The focus has been in the production of large numbers of PhD’s. • The system should better support high-quality PhD studies and in particular post-doc careers of young scientists. • This problem has been recognized also in the governmental research institutes.  FGFRI will create a system where post-docs can be recruited more efficiently (e.g. through open international calls).

11. (b) Effective use of long-term data sets • The long-term data sets collected by institutes are essential for their: • research activities, and • management services. • Effective and integrated use of these data sets is of fundamental importance in maximizing the understanding of ecosystem processes. • Co-operation is getting stronger and stronger with several universities and institutes, and data sets are increasingly used in shared projects. • We are willing to develop approaches to better share the data. • Free availability of data through internet demands proper technology, well designed rules, and perhaps also mental revolution.

12. (c) Inventory of the current data series • The importance of long-term data sets is constantly increasing.  • An assessment should be made of the reliability and usefulness of the current data sets. • In addition, the role of long-term monitoring conducted by different institutes and universities should be better synchronized, secured and strengthened. • The Finnish government is currently reforming the organization and funding of the governmental research institutes: • every possible effort should be made to ensure the continuation of the most valuable data series, and to further improve their quality.

13. (d) Some other recommendations • There should be more research on flowing waters, including sophisticated ecology-based approaches of restoration. • Research on socio-economic aspects needs substantial strengthening. • There should be more integration among predictive modellers and experimental scientists (model validation).  There has been apparent progress in this area during the last 10 years (e.g. BIREME-programme funded by the Academy of Finland). • A stronger and wider international cooperation should be fostered by Finnish universities and institutes.  A more active participation in international research projects (e.g. EU-funded) would be one potential way to dealt with this demand.

14. Conclusions • The demands for high quality and comprehensive aquatic research is dramatically increasing (Climate change, Baltic Sea, etc). • The recent policy of reduction the resources of research institutes creates major challenges to maintain the current high standards, not to mention to improve the standards. • This evaluation will help us to prepare for the future challenges. • It is good to know where we stand, and where to improve! • Comparison of earlier evaluations and the recent one indicates that we have responded successfully to majority of the recommendations given in the previous evaluations. • We will do the same also concerning the recent evaluation - it is in our best interest.

15. Our warmest thanks to You for the valuable work done in favour of the water research in Finland!