1 / 14

System Review of Evaluation in German Development Cooperation

System Review of Evaluation in German Development Cooperation. DAC Network on Development Evaluation meeting, 18 November 2008 Michaela Zintl, Head of Evaluation and Audit Divison, BMZ. Rationale. First review in 1998 / implementation of recommendations monitored in 2001

abel-charles
Download Presentation

System Review of Evaluation in German Development Cooperation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. System Review of Evaluation in German Development Cooperation DAC Network on Development Evaluation meeting, 18 November 2008 Michaela Zintl, Head of Evaluation and Audit Divison, BMZ

  2. Rationale • First review in 1998 / implementation of recommendations monitored in 2001 • Significant changes since: • International level: MDGs & Paris Declaration; new developments also in the field of evaluation • National level: changes in the budget code; „joint up“ German development cooperation

  3. Purpose • ...to assess whether the evaluation system of German development cooperation performs adequately with a view to strategic and conceptual demands and in terms of organisational, managerial and methodological benchmarks (DAC norms and standards, international good practice) • ...to develop recommendations for improvements of the system

  4. Scope : evaluation systems of • BMZ (Ministry) • Six implementing agencies (KfW, GTZ, ...) and the private sector arm (DEG) • 12 CSOs receiving funds from BMZ • further 30 CSOs (online questionnaire) ...covering 80% of bilateral aid funded by BMZ

  5. Methodology • Guided self-assessment • Document analysis (including assessment of evaluation reports) • Face to face interviews with staff and management of all organisations involved, external experts and MoP: 170 interviews

  6. Caveat: • Not included: • Partner countries‘ views • DC managed by other federal ministries and federal states • Voluntary contributions to international organisations • „Light“ review of evaluation reports

  7. Findings: Independence and credibility + Significant improvements since last review + Most organisations acknowledge DAC norms and standards - Evaluation underfunded almost everywhere - Coverage appears wanting (insufficient data) - Lack of ex post evaluations, impact assessments and comprehensive comparative studies

  8. Findings: Quality + Has generally improved, due to • Acceptance of standards & training • Results orientation • Improvement in process quality - Insufficient methodological awareness – mix of methods more rhetoric than practice; - Almost no impact evaluations - Evaluation knowledge not considered a priority for selection of staff and consultants

  9. Findings: Participation o No significant changes since last review - Evaluations continue to be donor centered except for some NGOs - Insufficient use of local independent consultants - (Too) little evaluation capacity building

  10. Findings: Utility + Learning is usually the primary objective + Evaluations are considered valuable by intended users o Mostly used for management of individual programms – little conceptual learning - Little inter-institutional learning - Focus on learning by donor agencies not partners

  11. Conclusions I: Improvements on many dimensions but challenges remain • Institutional independence increased but insufficient use of external consultants • Credibility increased because of better quality but limited by lack of transparency • Quality improved but limited range of methods used • Participation: no significant improvement • Utility: internal and instrumental learning: good and improved; inter-ageny and conceptual learning: poor

  12. Conclusions II: System development • Heterogenity between agencies has rather increased over time • System tilt between BMZ and implementing agencies • Steering capacity of BMZ too low • Most pressing problem: overcoming institutional fragmentation

  13. Recommendations • General recommendations along the various dimensions in synthesis report – specific recommendations in case study reports • On the system level: • Massive strengthening of BMZ • Independent evaluation agency (SADEV model) • Independent evaluation advisory board (DFID model)

  14. Lessons learned/ follow-up: too early to say but in any case • Combination of external experts and peers essential • Task to review 20 organisations is overwhelming • Partly a very political process, partly benign neglect • Change triggered already during the process in almost all organisations, although some prefer to say otherwise

More Related