1 / 12

VIII Konferencja Ewaluacyjna Meta-Evaluation Some Reflections Veronica Gaffey

VIII Konferencja Ewaluacyjna Meta-Evaluation Some Reflections Veronica Gaffey. Meta-Evaluation Some Reflections Evaluation Conference, Warsaw 12/13 November 2012. Veronica Gaffey Head of Evaluation DG for Regional and Urban Policy European Commission. RegionalPolicy. Presentation.

nimrod
Download Presentation

VIII Konferencja Ewaluacyjna Meta-Evaluation Some Reflections Veronica Gaffey

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. VIII Konferencja Ewaluacyjna Meta-Evaluation SomeReflections Veronica Gaffey

  2. Meta-EvaluationSome ReflectionsEvaluation Conference, Warsaw 12/13 November 2012 Veronica Gaffey Head of Evaluation DG for Regional and Urban Policy European Commission RegionalPolicy

  3. Presentation Meta-Evaluation – just how ambitious do we want to be? A more modest approach focused on the accumulation of knowledge Appropriate level – EU/MS or both? The role of meta-analysis in ex ante evaluation Elements in the new regulation to support accumulation of knowledge, moving towards meta-analysis Regional Policy

  4. 1. Meta Evaluation What is it? Aim to improve decision making through systematic reviews on the effects of intervention Examples: Cochrane Collaboration – Health – since 1992 Campbell Collaboration – education, crime and justice, social welfare – since 1999 Is our evidence base sufficiently strong? Issues of internal & external validity RegionalPolicy

  5. Our evidence base Of 266 completed evaluations examined by Expert Network in 2011, only 34% concerned effects; 66% implementation issues Pilot review process for DG REGIO finds few evaluations of Cohesion Policy of really good quality Difficult to judge quality of evaluations without terms of reference, budget and knowledge of context RegionalPolicy

  6. 2. Accumulation of Knowledge Some experience here: Meta-analysis of 200 evaluations across Europe prepared for Polish Presidency Conference on Evidence based Cohesion Policy Forthcoming Regional Focus on What we have learned from counterfactual evaluations of innovation and enterprise support (Daniel Mouqué) Internal Analysis on Transport and Environmental Infrastructure – likely to be transformed into guidance Insights rather than proofs; questions rather than prescriptions Regional Policy

  7. 3. Appropriate Level for Accumulating Knowledge Experience shows it can be done at EU level, if Terms of Reference and budget are public Context and method are explicit "Why it works?" is addressed as well as "Does it work?" Language issue can be overcome DG REGIO starting to build a library of MS evaluations Also at Member State level, though Difficult to generate critical mass Regional Policy

  8. 4. Meta-Analysis in Ex Ante Evaluation Ex ante evaluation should be theory and evidence based Testing the logic of intervention is at its heart – evidence from literature and other evaluations should be used Unpick the triangle in the next slide…. Regional Policy

  9. 5. Elements in the New Regulation Clearer articulation of objectives with associated result indicators and baselines – success can be judged Requirement for an evaluation plan Requirement for an impact evaluation of each priority during the programming period Summary report in 2021 – a meta-evaluation? Regional Policy

  10. Possible Next Steps… We continue to encourage more impact evaluations – using different and combined methods Competition/Prize for good Terms of Reference? Structured exchanges of experience – based on themes Building communities of practice – involving public administrations & academic experts Regional Policy

  11. Thank you for your attention RegionalPolicy

More Related