Planning and Running R&D Projects: Specifics, Ranking Factors, and Evaluation Methods

1. IFI8109 Planning and Running Research and Development ProjectsSpecifics of R&D projects

2. Plan • Presentations of the homework to the previous class – 5-7 minutes each (those who have not done it yet). • Comments to the critical aspects in evaluation of R&D projects. • Specifics of research projects. • Specifics of development projects. • Practices of taking into account of evaluation criteria. • Preconditions for applying for a R&D project. • Planning of research projects.

3. Specifics of research projects

4. Ranking of factors • Existence of high level publications (level and number). Why? • The value of h-index: the largest natural n, such that the author has n 1.1 level publications and each on them is cited more than n times. NB! Google Scholar (http://scholar.google.com/) can be used for finding the h-index. • Actuality (topicality) of the research problem, elaboration of the project (including the analysis of prior research, and correctness and readability of the text), and the relevance of methodology. • Experience in completing similar projects. • Supervision of graduate students.

5. Research projects – additional aspects • Quality of research questions and/or hypotheses. • Expected amount and level of new knowledge, for example: • Answering previously raised open/unsolved problems • Elaboration of a new framework, model, methodology or algorithm • Improvement of some framework, model, methodology or algorithm • Identifying relations between the different phenomena • Showing inaccuracy of some previous results (of other researchers). • Earlier cooperation between the project team members. • Consistency of the research field of institutional development and national development plans. • International and/or internal cooperation.

6. Specifics of development projects

7. Ranking of factors • Impact of the project – both nationally and internationally – and the width of the application area of the project’s outcomes. • Project management, including reality and measurability of project’s objective/outcome, quality of the action plan and budget, experience of the team in the project area. • Technological level and innovativeness of a project, including conformity of the project’s conception to the objective.

8. Development projects – additional aspects • Indirect/directly not measurable impact (for example on economic development or on the development of sectoral technologies). • Aspects related to implementation of the project’s outcome, including for example: • capability of implementation • Intellectual property issues. • Sustainability of the project, including the market demand and potential export potential and social impact.

9. Methods for taking into account the evaluation criteria

10. Consideration of formal criteria • Check out the documentation of the measure and follow them strictly. • Check out the evaluation criteria and take them into account in composing the project proposal. Ask if necessary! Example of a failure: Humboldt Foundation. • Determine the critical success factors of the project and your bottlenecks (biggest problems). • Pay special attention to the bottlenecks when composing the project proposal, involving additional competence if necessary.

11. Consideration of non-formal criteria • Try to get clarity on current assessment practice: • Consult with members of the decision-making bodies (in particular about the aspects that have greater weight in evaluation). Example: the age of PI-s. • If possible, study the previous project proposals and their reviews. • If possible, participate in decision-making bodies. • Identify your greatest strengths (in comparison with other potential applicants) and how the project can benefit of it. • Base on your strengths, integrating their description and their need for completing the project in the application text.

12. Preconditions for applying for a R&D project(conditions that should be satisfyied before initiating a R&D project)

13. Basic principle • Support is given to the teams that already have achieved a high (international) level, not for reaching this level. • Workable strategy for junior researchers: • Start as a member in a group of high level researchers, conducting joint studies and publishing joint articles.

14. Preconditions for applying a research project • Satisfaction of formal qualification requirements. Why? • Dealing with a widely acknowledged (yet unsolved) problem. • Existence of a novel original idea for solving the problem. • Project financing probability is large enough, i.e. the formal requirements as compared to potential competitors are well met. NB! In research projects - as opposed to development projects - risk analysis is in general not required (highly innovative research is almost always risky).

15. Initation of a research project – recommendations • Be a (co-)author in as many as possible high level articles and a member of project teams. • Before applying a research project, check out the competence indicators of previous successful applicants. • Follow strictly the formal requirements set for project proposals.

16. Next ...

17. Example: Estonian classification of research publications • Categories that are counted for doctoral students: • 1.1 – Scholarly articles indexed by Thomson Reuters Web of Science and/or published in journals indexed by ERIH (European Reference Index of the Humanities) categories INT1 and INT2. • 1.2 – Peer-reviewed articles in other international research journals with an ISSN code and international editorial board, which are circulated internationally and open to international contributions; articles of ERIH category NAT. • 3.1 – Articles/chapters in books published by the publishers listed in Annex (including collections indexed by the Thomson Reuters Conference Proceedings Citation Index, see http://www.etag.ee/teaduskirjastused/). • Vt https://www.etis.ee/otsingud/classification.aspx

18. Example: trivial hypothesis • The field of pedagogical pattern language is broad and well-developed, but the proposal gives little indication as to even the kinds of patterns that might be suitable for the language they wish to create. It is not much of a scientific hypothesis to say that "Pedagogical pattern language is an effective, valid and reliable instrument for analysing and formalizing e-learning processes."

19. Example: calculation of h-index (Google Scholar) • On equalizer-flat and pullback-flat acts, 54 citations • Purity in the category of M-sets, 24 citations • Monoids over which all flat cyclic right acts are strongly flat, 19 cit. • Soft ontologies, spatial representations and multi-perspective explorability, 14 citations • Flatness properties of monocyclic acts, 11 citations • On Noetherian and finitely presented polygons, 7 citations • Congruence Compact Acts, 6 citations • Morita duality for monoids, 6 citations • Hereditary endomorphism monoids of projective acts, 4 citations

20. How to measure impact? • In monetary terms (tools for cheaper production). • Time savings (quicker production). • New functions/procedures possible. • Greater simplicity • Higher quality • Broader target/user group • …