Evidence- based evaluation

1. Evidence-basedevaluation RNDr. Jan Vozáb, PhD partner, principal consultant Berman Group

2. Objectivesofthe evidence-basedevaluation • To increase impact and value added of the Cohesion Policy interventions • To increase effectiveness and efficiency of interventions in terms of achieving results that make real (and desirable) changes • To provide better and more reliable information for strategic management of OPs

3. Key issues and dilemmas • Evidence is always part of its context – explanation of findings is part of the evidence-based evaluation • The more complex, complicated or integrated is the intervention, the more difficult is to obtain reliable, strong evidence of its effects • The better is the evidence for measuring of effects, the more difficult is to cause them by the intervention itself

4. Evaluators´ position Real issues Real effects Evaluation MAs intentions Interventions / Operations Outcomes of interventions

5. Programming issues related to evidence-based evaluation • Problem and needs identification • Definition of objectives • Planning of operations • Set up of targets and indicators structure Implications for evaluations: • Difficulties to link outcomes to the objectives and needs • Difficulties to find evidence of real effects of operations (although they exist) Intervention logic

6. Implementation issues related to evidence-based evaluation • Random or non-targeted selection of projects • Week coordination of operations within OP • Feedback on outcomes for management purposes • Costs to track evidence on results/impacts Implications for evaluation: • Complicated establishing of evidence on outcomes from available data sets • Lack of suitable data for evaluation – monitoring data only

7. Methodological issues related to evidence-based evaluation • Formulation of evaluation questions • Use of representative data • Explanation of findings in proper context • Deriving conclusions, providing recommendations Implications for evaluation: • Evidence based on intervention logic • Availability of data • Combination of qualitative and quantitative evidence

8. Recommendations for improving evidence-based evaluation • To make sure that programming is sufficiently evidence-based as a prerequisite for evaluation • To strengthen vertical and horizontal intervention logic within and among OPs and partnership contract • To make sure that operations aim at causes of problems and are sufficiently coherent and focused • To strengthen management capacity of MAs: • To manage operations and deliver interventions • To take more responsibility for results and wider effects of interventions (not only for indicators/targets)

9. Recommendations for improving evidence-based evaluation • To strengthen analytical capacity and skills of evaluators • To use evaluation in a better way to support decision-making: • Small, targeted, ad hoc evaluations • To adjust monitoring systems for evaluation purposes • To strengthen links between evaluators and professionals in thematic fields • To establish or strengthen feedback on incorporation of evaluators´ findings into OP management

10. Thank you for your attentionJan Vozábvozab@bermangroup.cz