Agenda

1. ROLE OF EVALUATION IN PREPARINGAND IMPLEMENTING OF THE 2014–2020PROGRAMMING PERIOD 18thJanuary 2012, Prague

2. Agenda Role of evaluation in preparing and implementing of the 2014-2020 programming period • 1. Introductory Word10:00 – 10:05 • 2. EC requirements in the field of evaluation in 2014+10:05 – 11:00 • 3. Czech evaluation experience 11:00 – 12:10 • 12:10–12:30 – Break • 4. Recommendations on how to focus on results12:30 – 13:10 • 5. Concluding Summary13:10 – 13:15 • 13:15– 14:30 – Lunch

3. EC requirements in the field of evaluation in 2014+VeronicaGaffey, Acting Director Directorate C, DG Regio, EC Role of evaluation in preparing and implementing of the 2014-2020 programming period

4. EC requirements in the field of evaluation in 2014+ Discussion Role of evaluation in preparing and implementing of the 2014-2020 programming period

5. Czech evaluation experienceVít Šumpela, National Coordination Authority, Ministry of Regional Development Role of evaluation in preparing and implementing of the 2014-2020 programming period

6. Role of evaluation in preparing and implementing of the 2014-2020 programming period National Strategic Reference Framework in CZ OP with management authority in CZ OP with management authority abroad 6 6 6

7. Czech evaluation experience Role of evaluation in preparing and implementing of the 2014-2020 programming period • Setting of the physical progress monitoring system in 2007–2013 • Indicator System • = a comprehensive and coherent system of indicators at the level of an operational programme, which includes indicators from the lowest project level to the programme level. Indicator’s systems of operational programmes are connected to the indicator system of the NSRF. • National Codebook of Indicators (NCI) • A tool coordinating development of indicator systems. • Contains indicators with clear definition, measurement unit and coding which are concerted from the technical and methodology points of view.

8. Role of evaluation in preparing and implementing of the 2014-2020 programming period Czech evaluation experience Number of indicators in the OP’s

9. Czech evaluation experience Role of evaluation in preparing and implementing of the 2014-2020 programming period • Setting of the physical progress monitoring system in 2007–2013 • NCI Optimisation • Harmonization of measurement units, names and codes of certain indicators. • Binding methodology (incl. aggregation maps). • Firmly-set rules based on the Central Monitoring Methodology reflected by the monitoring system are correct. • Certain output indicators lack a strong or have no link to results. • Currently, the existing indicator system can only be minimally used for evaluating success and efficiency of interventions. • Set also qualitative indicators (CIE). • Start developing an indicator system early!

10. Czech evaluation experience Role of evaluation in preparing and implementing of the 2014-2020 programming period • Links of indicators to intervention objectives • Evidence Based Cohesion Policy • Set objectives of the strategy and the indicator system properly, including input and target values of individual indicators when preparing the strategy and programme documents. • Experienceof2007-2013: • All Member States set high objectives in 2007–2013, • Result indicators are based on statistical data which are not available at the time when they are needed, • Given issues are impacted by many factors some of which cannot be influenced from the viewpoint of interventions.

11. Czech evaluation experience Role of evaluation in preparing and implementing of the 2014-2020 programming period • Theory of change • It is recommended to define the basic theory of change for the future period (What do we want and can change?) and already when developing a strategy to set up the mechanism of its implementation (How do we want to achieve it?). • Ex-ante evaluation should include verification of the proper set-up of the indicator system and its evaluation (How to verify that we were successful?). • These questions are the basic parameter for applying the principle of Evidence Based Cohesion Policy.

12. Role of evaluation in preparing and implementing of the 2014-2020 programming period New logical framework setting the strategy and indicator system for 2014+

13. Czech evaluation experience Role of evaluation in preparing and implementing of the 2014-2020 programming period Setting of the indicator system for the 2014–2020 programming period It is proposed to use NCI and mandatory list of output and result indicators. When and how? Determine a specific change which we want and can achieve through interventions. In relation to this required change, identify a limited number of clearly targeted activities to be supported by interventions.

14. Czech evaluation experience Role of evaluation in preparing and implementing of the 2014-2020 programming period • 3. Determine indicators to monitor performance for each defined activity: • Set the output and result indicators for the level of OP/priority/activity/project, their definition, measurement unit, target value, monitoring methodology and data source + clear relation with the supported activity. • The output indicators must be monitored from the project level. The methodological setting must comply with the aggregability rule. • The interventions should be evaluated on the basis of the result indicators. The values of the result indicators must show a change of the status. Setting in absolute values is recommended. • The statistical (context)indicators must be set on the basis of regularly available data from the statistical sources, e.g. CSO.

15. Czech evaluation experience Role of evaluation in preparing and implementing of the 2014-2020 programming period • Setting of the indicator system for the 2014–2020 programming period • Important is: • to set the indicator system in parallel with preparation of the coming programming period of 2014–2020. • to know how to monitor all the interventions and how to evaluate their results. • to use qualitative evaluation method (beside quantitative). • to set the strategy from the very beginning in such a way so as to be able to evaluate it by means of appropriate indicators set in accordance with the aforementioned rules.

16. Czech evaluation experience Role of evaluation in preparing and implementing of the 2014-2020 programming period • Evaluation of interventions on the basis of indicators • The purpose of the indicator system is to ensure proper setting of the strategy and evaluationability of achieved results. • We must support such projects, the outputs and results of which will directly lead to fulfilment of the set strategy. • The on-going evaluation based on the above recommendations for indicators may become a key instrument for successful management of interventions and programme implementation.

17. Czech evaluation experienceVladimír Kváča, Ministry of Labour and Social Affairs Role of evaluation in preparing and implementing of the 2014-2020 programming period

18. Czech evaluation experienceDiscussion Role of evaluation in preparing and implementing of the 2014-2020 programming period

19. Recommendations on how to focus on results Jan Vozáb, Evaluation expert Role of evaluation in preparing and implementing of the 2014-2020 programming period

20. Recommendations on how to focus on resultsDiscussion Role of evaluation in preparing and implementing of the 2014-2020 programming period

21. Concluding Summary Daniel Braun, Deputy Minister, Ministry of Regional Development Role of evaluation in preparing and implementing of the 2014-2020 programming period