Proposal evaluation process in FP7

1. Proposal evaluation process in FP7 Moldova – Research Horizon 29 January 2013 Kristin Kraav

3. Evaluation process Individual evaluation Panel meeting Consensus meetings Commission pre-actions Commission Follow-up

4. Commission pre-actions Individual evaluation Panel meeting Consensus meetings Commission pre-actions Commission Follow-up

5. Commission pre-actions • Appointment of call coordinator and coordinators for the topics • Selection of the independent experts for the Call • FP7 experts database (https://cordis.europa.eu/emmfp7) • personal head-hunt • Proposals eligibility check • eligibility criteriamaybe budget, number of partners, duration … • Formation of the expert panels for each topic of the call • Assignment of proposals to chosen experts

6. Individual assessment Individual evaluation Panel meeting Consensus meetings Commission pre-actions Commission Follow-up

7. Individual assessment • Remote evaluation period ~4-6 weeks • Everything happens in online system RIVET • experts can access only assigned proposals • Max 10 proposals per expert, min 3 experts per proposal • Experts have to make their decisions based on: • proposal • Callbackground documents • evaluation criteria • Result – Individual Assessment Report • experts personal opinion • basis for the discussions in the next stage

8. The evaluation criteria • Three evaluation criteria (Cooperation and Capacities – People and Ideas differ significantly): • Scientific and/or Technological excellence • Quality and efficiency of the implementation and the management • The potential impact through the development, dissemination and use of project results

9. Scoring the proposals • Each proposal is evaluated against all criteria and scored between 0 and 5 • Threshold per criteria – 3 • Overall threshold per proposal – 10 • This means – if any of the criteria under 3 points or the total scoring of the proposal under 10 points, the proposal will be placed to the list of rejected proposals for reason  under the threshold

10. Interpretation of the scores • 0 – The proposal fails to address the criterion under examination or cannot be judged due to missing or incomplete information. • 1 – Poor. The criterion is addressed in an inadequate manner, or there are serious inherent weaknesses. • 2 – Fair. While the proposal broadly addresses the criterion, there are significant weaknesses. • 3 – Good. The proposal addresses the criterion well, although improvements would be necessary. • 4 – Very good. The proposal addresses the criterion very well, although certain improvements are still possible. • 5 – Excellent. The proposal successfully addresses all relevant aspects of the criterion in question. Any shortcomings are minor.

11. Consensus meetings Individual evaluation Panel meeting Consensus meetings Commission pre-actions Commission Follow-up

12. Panel of experts

13. Consensus meetings • 1 week for face-to-face meetings, 12 hour per day • Consensus meeting • 1.5 hour meeting per proposal gathering all experts who evaluated it, moderated by EC staff-member • basically restart of evaluation • has to end withconsensus on scores and comments for each of the criteria → Consensus Report • On the bases of discussions rapporteur (one of the evaluators) drafts the first version of Consensus Report

14. Consensus meetings → ESR • First draft of CR turns into Evaluation Summary Reportthrough: • co-operation of all experts involved • comments from Call coordinator and topic coordinator • help of language editor • ~4-5 day parallel process for all proposals submitted under the topic • If the CM does not end with consensus: • additional experts from the panel will evaluate it • new CM is called • If still no consensus → total restart of the process with new panel of experts

15. Panel meeting Individual evaluation Panel meeting Consensus meetings Commission pre-actions Commission Follow-up

16. Panel meeting → proposed ranking list • Drawing up the ranking list of the topic • Comparison of scores and ESR • Final marks and comments for each proposal • Suggestions on order of priority, clustering, amendments, etc. • Simple process if the ranking is clear • Tricky process if many proposals have equal score and all of them can not be funded • Priority order ofcriteriaincaseofequalscores • If all scores equal, experts have to decide which score is stronger and support it with relevant arguments

17. Commission follow-up Individual evaluation Panel meeting Consensus meetings Commission pre-actions Commission Follow-up

18. Commission follow-up • Draw up final ranking lists • decisions on proposals selected for funding • decisions on rejected proposals • Information and data to the Programme Committee • PC formal agreement needed for financing proposals with budget over 600 000€ • Independent Observers’ report • Invitation to Contract Negotiation and Evaluation Summary Reports sent to coordinators • Contract negotiations

19. Evaluation process Individual evaluation Panel meeting Consensus meetings Commission pre-actions Commission Follow-up

20. Evaluation process – lessons learned Individual evaluation Panel meeting Consensus meetings Commission pre-actions Commission Follow-up

21. Process of evaluation • Evaluationis a deconstruction of proposals • Comparison of different elements of proposal against each other • overall aim → steps to achieve it (activities) → expertise (partners) for managing the process → allocatedresources • Everything should be written as clearly and shortly as possible – still all required parts of Part B have to be there • “A picture is worth more than 100 words” – tables, graphs, schemes help immensely • Do not expect evaluators to assume things

22. Remember: • Evaluators are humans , they • come from different professional and ethnical backgrounds; • have different beliefs of right and wrong; • speak different languages; • usually are under time pressure • Respect the evaluators: • follow the structure • edit your proposal to eliminate typos and other mistakes • make the proposal legible: font and font size, structured text, test that all graphics can be read in b&w

23. Conclusions • Fit to the call : read carefully the call topic • Be outstanding on the Scientific and Technology point of view but do not underscore the other criteria • Be credible • Demonstrate an EU added value • Respect the rules : read the Work Programme and GuideforApplicants