PROPOSAL WRITING AND EVALUATION: Guidance for applicants and evaluators for Calls in Horizon2020

1. PROPOSAL WRITING AND EVALUATION: Guidance for applicants and evaluators for Calls in Horizon2020 Sergey Mikhalovsky, Nazarbayev University, Astana smikhalovsky@nu.edu.kz University of Brighton, UK s.mikhalovsky@brighton.ac.uk

2. My recent experience with research funding Co-ordinator, Principal Investigator or Team leader: European Union: FP6-MATISS (2006-2010) FP7-MONACO-EXTRA (2008-2012) FP7-OncoNanoBBB (2012-2015) FP7-ABREM (2010-2014), FP7-FRESP (2009-2012) FP7-Greenland (2009-2012) Interreg IIIA Stent (2005-2008) Interreg IVA Flax (2009-2012) TEMPUS III and TEMPUS IV (2005-2008, 2009-2012) UK: Technology Strategy Board - FullFlush (2010-2013), Department of Health (2008-2013), Medical Research Council (2012-2015), The British Council (2014-2015), MES of Kazakhstan

3. My experience with proposal evaluation FP5, FP6, and FP7 in nanotechnologies, environmental sciences and Marie Curie Programme (including Horizon 2020) Erasmus Mundus I and II, Lead Expert in Life Sciences, Erasmus+ TEMPUS II and III, INTAS, EPSRC (UK), BBSRC (UK), TSB (UK), national programmes for Russia, Austria, Cyprus, France, Montenegro, USA and Kazakhstan

4. What about Kazakhstan? Kazakhstan is on the list of International Cooperation Partner Countries (ICPC) And eligible for most Horizon2020 projects! A window of opportunity: BRICS countries are NOT eligible for EU funding in Horizon 2020!

5. Where to find information about EU funding? • Information about Horizon 2020 and ALL OTHER European Union initiatives and funding can be found on the website: • http://ec.europa.eu/research/participants/portal/desktop/en/home.html

7. You should register on this site as an individual researcher:

8. Click EXTERNAL

9. In future you would be able to access information via this site

11. Here you will find all information about Horizon 2020 and its calls

13. Rules on submission & evaluationBasic principles • Excellence. Projects selected for funding must demonstrate a high quality in the context of the topics and criteria set out in the calls. • Transparency. Funding decisions must be based on clearly described rules and procedures, and applicants should receive adequate feedback on the outcome of the evaluation of their proposals. • Fairness and impartiality. All proposals submitted to a call are treated equally. They are evaluated impartially on their merits, irrespective of their origin or the identity of the applicants.

14. Rules on submission & evaluationBasic principles • Confidentiality. All proposals and related data, knowledge and documents communicated to the Commission are treated in confidence. • Efficiency and speed. Evaluation, award and grant preparation should be as rapid as possible, commensurate with maintaining the quality of the evaluation, and respecting the legal framework. • Ethical and security considerations: Any proposal which contravenes fundamental ethical principles, or which fails to comply with the relevant security procedures may be excluded at any time from the process of evaluation, selection and award

15. The Three MainReference Documents 1 - Rules on submission and evaluation • This is the common and official reference for Horizon2020 rules for submission of proposals, and the related evaluation, selection and award procedures. 2 – Information (in FP7 - Guide) for applicants • The Guide for Applicants contains the essential information to guide proposers through the mechanics of preparing and submitting a proposal. • All proposals shall contain a Part A (administrative forms) and a Part B (proposal description). Indications about the content and issues to be addressed are described in the Guides for Applicants • Please make sure that you read the “Guide for applicants” that corresponds to the funding scheme for the topic.

16. The Three PrincipalReference Documents • 3 - The work programme: three complementary documents: • Work Programme - General Introduction • 2014-2015 Work Programme: includes the topic’s description and criteria against which the proposals will be assessed. • Cooperation Work Programme - General annexes • Annex 1: List of International Co-operation Partner Countries (ICPC) • Annex 2: Eligibility and Evaluation Criteria for Proposals • Annex 3: Forms of Grant and Maximum Reimbursement Rates for Projects Funded • Annex 4: General activities

17. Independentexperts • Expert evaluators are at the heart of the Horizon2020 system • Provide independent, impartial and objective advice to the Commission • they represent neither their employer, nor their country! • Significant funding decisions will be made on the basis of their advice • They can also add value to projects through your comments and suggestions • The integrity of the process is crucial • They should follow the Code of Conduct annexed to the appointment letter

18. Who evaluates? • Selected from a wide pool from a database, on the basis of keywords • Minimum 3 evaluators per project • Selected per call • Replace about ¼ in any given area annually • Sign confidentiality and conflict of interest declarations • Names published after evaluation (though not at call or proposal level) • Target at least 40% female • Mix of geographical location and background • To register: https://cordis.europa.eu/emmfp7 FP7 – How to apply

19. Full Proposal Proposal forms Overview of the Evaluation Process “remote” may be “remote” Submission Individual reading Consensus Panel Finalisation Final ranking list Evaluators Evaluators Evaluators Rejection list Criteria Criteria Criteria Proposals in suggested priority order Eligibility COMMISSION COMMISSION Role of experts

20. ProcessFor each proposal: May be “remote” and / or central “remote” Proposal X copy 1 IER expert 1 Consensus meeting Proposal X copy 2 CR 3 experts IER expert 2 Proposal X copy 3 IER expert 3 Note: There may be more than 3 evaluators IER=Individual evaluation report CR=Consensus Report

21. ProcessEthical Issues • The Consortium is asked to submit drafts of Information Sheet and Consent Form but does not need to submit copies of legislation • Proposals should comply with • fundamental ethical principles • relevant security procedures • … or be excluded from the process • Check if the proposal has in fact ethical issues • If yes: tick ethical issues box in the CR • Prepare Ethical Issues Report

22. ProcessEthical Issues Although the main focus is on the ethical dimension (e.g. human rights and protection of human beings, animal protection and welfare, data protection and privacy, environmental protection, malevolent use of research results), Horizon 2020 will also look at ‘research integrity’ issues (e.g. fabrication, falsification or plagiarism, including misrepresenting credentials and authorship improprieties).

23. ProcessEvaluating a proposal • Three guiding principles: • Objectivity • each proposal is evaluated as it is written • Accuracy • The judgment is made against the official evaluation criteria, and nothing else • Consistency • The same standard of judgment applies to each proposal

24. ProcessThe evaluation criteria • Criteria adapted to each funding scheme and each thematic area • specified in the work programme • Three main criteria: • Excellence (relevant to the topic of the call) • Soundness of the concepts; Clarity and pertinence of the objectives, Credibility of the proposed approach; Extent that proposed work is ambitious, has innovation potential, and is beyond the state of the art (e.g. ground-breaking objectives, novel concepts and approaches

25. The evaluation criteria (cont) • Impact • The extent to which the outputs of the project should contribute at the European and/or International level to: The expected impacts listed in the work programme under the relevant topic; Enhancing innovation capacity and integration of new knowledge; Strengthening the competitiveness and growth of companies by developing innovations meeting the needs of European and global markets; and, where relevant, by delivering such innovations to the markets; Any other environmental and socially important impacts; Effectiveness of the proposed measures to exploit and disseminate the project results (including management of IPR), to communicate the project, and to manage research data where relevant; Strengthening the competitiveness and growth of companies by developing innovations meeting the needs of European and global procurement markets • Quality and Efficiency of Implementation • Individual participants and consortium as a whole • Allocation of resources

26. The evaluation criteria (cont) • Quality and Efficiency of Implementation Coherence and effectiveness of the work plan, including appropriateness of the allocation of tasks and resources; Complementarity of the participants within the consortium (when relevant); Appropriateness of the management structures and procedures, including risk and innovation management;

27. ProcessEvaluation Criteria Applicable to ALL funding schemes

28. ProcessEvaluation Criteria Collaborative projects

29. Process Funding schemes • Collaborative projects Support to research projects carried out by consortia with participants from different countries, aiming at developing new knowledge, new technology, products, demonstration activities or common resources for research. The size, scope and internal organisation of projects can vary from field to field and from topic to topic.Projects can range from small or medium-scale focused research actions to large-scale integratingprojects for achieving a defined objective Projects may also be targeted to special groups such as SMEs, Specific International Co-operation Actions, etc.

30. ProcessProposals that are onlypartly in scope • Note: The “S/T quality” of a proposal (first criterion) is evaluated to the extent that the content is relevant to the topic(s) addressed by the call • E.g. If a proposal is only marginally relevant, or if only one work package is relevant, the evaluator must downgrade the score – no matter how excellent is the science! • Relevance to the objectives of the call is also considered under “Impact” (third criterion)  • In relation to the sub-criterion “contribution to expected impacts listed in the work programme”

31. ProcessProposal scoring:Interpretation of the scores • 0: The proposal fails to address the criterion under examination orcannot be judged due to missing or incomplete information • 1: Very poor. The criterion is addressed in a cursory and unsatisfactory manner. • 2: Poor. There are serious inherent weaknesses in relation to the criterion in question. • 3: Fair. While the proposal broadly addresses the criterion, there are significant weaknesses that would need correcting. • 4: Good. The proposal addresses the criterion well, although certain improvements are possible. • 5: Excellent. The proposal successfully addresses all relevant aspects of the criterion in question. Any shortcomings are minor.

32. ProcessProposal scoring • Each criterion is scored 0-5 • Marks can go from 0 – 5 in steps of 0.5, i.e half-marks are allowed • Experts are encouraged to use the whole range • Scores must pass thresholds if a proposal is to be considered for funding • Thresholds apply to individual criteria… • Threshold is 3 • …and to the total score • higher than the sum of the individual thresholds • Threshold is 10 • Note that to receive a mark of 5, a proposal does not have to be perfect. An excellent proposal can have minor shortcomings. • When writing comments in the IERs and Consensus Report, the severity of any weakness should be clearly stated, i.e. are they minor, moderate or significant

33. ProcessProposal scoring • Evaluate the proposal and conclude whether the proposal is • Excellent • Good • Fair • Poor • Very poor • Failing to address the criterion • Score the proposal accordingly

34. ProcessCommission Follow-up • Evaluation summary reports sent to applicants • “initial information letter” • Redress procedure • Draw up final ranking lists • Information to the Programme Committee • Contract negotiation • Formal consultation of Programme Committee (when required) • Commission decisions • Survey of evaluators • Independent Observers’ reports

35. ProcessRedress • Proposers can complain if they believe there have been shortcomings in the handling of their proposal, and that these shortcomings have jeopardised the outcome of the evaluation process. • The quality and consistency of the evaluation reports (ESRs), derived directly from the CRs, is paramount to minimise the redress procedures

36. Tips on writing a successful proposal • An interesting and innovative idea • Strong consortium • Experienced co-ordinator • Complementarity of skills and expertise • Perfect matching between the proposal and the call • At least 6 months to prepare a proposal • European added value • Sustainability after the end of the project • Dissemination strategy FP7 – Tips

37. Please remember – evaluators judge your proposal at their face value, i.e., on the basis of the text submitted for evaluation. They only use additional sources if they need to check something you said in the proposal. If, when reading their evaluation, you think they did not understand you, it is YOUR fault

38. Why? • Good Money – typically 200,000-300,000 € per participating organisation for 2-3-4 years • More if you are the Lead Partner – all the money is distributed through you, which counts for your organisation – good for you and for your organisation – typically 1-2-3 million € for 2-3-4 years • Good Value for Money – typically 60% overheads

39. Why? (continued) • Fun – experience of different countries, cuisines and cultures, meeting new people • Knowledge transfer – opportunity to work in different labs, access to unique equipment and instruments, methods and techniques • Human mobility – you visit others and others visit you to exchange knowledge and experience • Fair competition and useful feedback

40. Why? (cont.) • Networking – you build up your own network • There may be unique expertise somewhere in Europe unavailable in this country • Career enhancement • Publicity – EC loves it and gives you lots of opportunities to do that (even if you don’t want it...) • Recruitment of high quality researchers • And you may even get a result...

41. What? • Incredibly diverse formats and degrees of participation: (i) individual fellowships – No age limit, very well paid! (ii) participation as a team – you can be paid or you can employ someone (iii) co-ordinator – the same as (i) and (ii) plus management costs plus permanent headache.

42. What? (cont.) Funding is available for salaries, overheads, travel, consumables, management, equipment and organisation of networking events.

43. How? Regularly monitor the funding opportunities Most obvious source of funding for regular monitoring are: (i) Horizon2020 - become a regular visitor on this site: http://ec.europa.eu/research/participants/ portal/page/home and become familiar with the structure of this programme. It is the main but not the only source of R&D funding in Europe

44. How? (cont.) (i) Be flexible with your idea of participation – sometimes there may be a Call which matches exactly what you want to do but that is unlikely. (ii) Identify potential areas of interest by monitoring previous Calls and - important! – analysing success rate in previous Calls. If the success rate is less than 10% - don’t go for it, unless you are a winner in a National Lottery or somebody else is prepared to do the main job of writing for you.

45. How? (cont.) • Try to get information about forthcoming Calls – it is possible to do so by attending consultations in Brussels or elsewhere, contacting your NCP (National Contact Point), getting information from an insider, etc. The earlier you know the contents of the future Call, the better your chances for success. When the Call has been announced, it is too late! You will have 2-3 months to submit a proposal from the date when the Call opens.

46. How ? (cont.) • There are also bottom-up Calls – these are my favourite – you can write about anything you like and the date of the new Call announcement is usually known well ahead, so you have plenty of time to think and prepare for the Call.

47. How ? (cont.) • There are also bottom-up Calls – these are my favourite – you can write about anything you like and the date of the new Call announcement is usually known well ahead, so you have plenty of time to think and prepare for the Call. • Erasmus+ programme offer in essence bottom-up Calls in Research&Education.

48. What to start with? • Formulate the idea, which fits a particular Call (probably several ideas to have a choice) • Identify potential partners – do a lot of networking, and do it all the time regardless of Calls. • Learn the Brussels speak – it takes time, but once you understand it – your chances for success are much higher.

49. What to do next? • You have to understand how to write a proposal • Become an expert evaluator for FP8 – Horizon 2020 it is called http://ec.europa.eu/research/horizon2020/index_en.cfm?pg=h2020-experts It will help you to understand the evaluation process and ultimately improve your proposal writing skills

50. What to do next? • Write a proposal – preferably not by yourself if it is your first one. • Write a proposal yourself if you have done this before • Seek advice and assistance of people with experience: colleagues who have got such grants, Research Office • Interact with other partners • Notify your line managers and finances about your intention to submit a proposal in advance