Evaluation of proposals

1. Evaluation of proposals Alan Cross European Commission

2. Evaluation: building on past experience…. - FP7 expert questionnaire

3. …adapting to Horizon 2020 • Less prescriptive calls; new types of proposal (multi-disciplinary and multi-sectoral; more emphasis on innovation and close-to-market) • Greater spread of expertise needed; standard briefing on all key issues… • Responding to 8 month time-to-grant imperative • Proposals evaluated as submitted - no recommendations for substantial modifications • Simplification: for applicants, experts, and for streamlined operations • Simpler forms/page limits; improved facilities for remote evaluation (incl. consensus); faster data transfer • Coherence across the progamme • A single set of guidelines (Vademecum chapter) for all DGs and Agencies

4. Overview of the Evaluation Process Evaluators Receipt of proposals Individual evaluation Panel Review Finalisation Consensus group Eligibility check Allocation of proposals to evaluators Individual Evaluation Reports (Usually done remotely) Consensus Report (May be done remotely) Panel report Evaluation Summary Report Panel ranked list Final ranked list

5. Evaluation Process Proposal Eligible proposal Individual Evaluation Report Expert Expert Expert Expert Expert Individual Evaluation Report Individual Evaluation Report Minimum 3 experts Individual Evaluation Report Individual Evaluation Report Individual evaluation Consensus group Consensus Consensus Report

6. Award criteria (Research & Innovation; Innovation Actions) • 1. Excellence • Clarity of the objectives; • Soundness of the concept, including transdisciplinary considerations; • Credibility of the proposed approach; • Ambition, progress beyond the state of the art. • 2. Impact: […] contribution to: • The expected impacts listed in the work programme • Enhancing innovation capacity and integration of new knowledge; • Strengthening the competitiveness and growth of companies by developing innovations meeting the needs of European and global markets; • …measures to, disseminate and exploit the project results,… communication. • 3. Quality and efficiency of implementation • Coherence and effectiveness of work plan, … allocation of tasks, resources; • Competences, experience and complementarity of the individual participants, as well as of the consortium as a whole; • Appropriateness of the management structures and procedures….risk management. First stage considerations in RED

7. Proposal scoring • Each criterion is scored 0-5 • half-scores allowed • whole range should be considered • scores must pass thresholds if a proposal is to be considered for funding • Thresholds apply to individual criteria… • Default threshold is 3 • …and to the total score • higher than the sum of the individual thresholds • Default threshold is 10 • Can vary from call-to-call! • First stage proposals: • first two criteria only • threshold = 4

8. Interpretation of the scores • 0 —The proposal fails to address the criterion or cannot be assessed due to missing or incomplete information. • 1 — Poor. The criterion is inadequately addressed, or there are serious inherent weaknesses. • 2 — Fair. The proposal broadly addresses the criterion, but there are significant weaknesses. • 3 — Good. The proposal addresses the criterion well, but a number of shortcomings are present. • 4 — Very Good. The proposal addresses the criterion very well, but a small number of shortcomings are present. • 5 — Excellent. The proposal successfully addresses all relevant aspects of the criterion. Any shortcomings are minor.

9. Weighting • Unlike FP7, for Innovation Actions and SME instrument… • Impact criterion weighted by factor of 1.5 • 'Impact' score given priority over 'excellence' when scores equal (the reverse applies in other types of action).

10. Selection of proposals All above threshold proposals in each topic are listed in descending order of overall score We select proposals starting from the top of the list, until the available budget is exhausted ••• 10