Fragile states: Perspectives from evaluations

1. Fragile states: Perspectives from evaluations AFDB Evaluation Week, 5th December 2012. Presentation by Beate Bull Evaluation Department, Norwegian Agency for Development Cooperation, Norad.

2. Norwegian development aid budget Side/Page

3. Ten largest recipients of aid Side/Page

4. Perspectives from peacebuilding /fragile states evaluations based on: • Evaluations done for and by the Norwegian development cooperation Agency, and reviewing others • A meta-review of evaluationsof support to statebuildingby Gravingholt, J. og Leininger, J. 2012 • OECD/DAC Guidanceonevaluatingpeacebuildingactivitiesin settings of conflictand fragility Side/Page

5. List of recent Norad(supported) evaluations of support to peace-building in settings of conflict and fragility • - “Evaluation of Norwegian Development Cooperation with Afghanistan 2001-2011”, (2012), http://www.norad.no/no/s%C3%B8k?q=aiding+the+peace; • «Pawns of Peace.Evaluation of Norwegian peace efforts on Sri Lanka 1997-2009 (2011)”, http://www.norad.no/no/resultater/publikasjoner/evalueringer/publikasjon?key=386346(2011) • “Aiding the Peace”: A Multi-donor Evaluation of Support to Conflict Prevention and Peacebuilding Activities in Southern Sudan 2005-2010. ITAD Ltd., United Kingdom.http://www.norad.no/no/s%C3%B8k?q=aiding+the+peace • Evaluation of Norwegian Peace efforts in Haiti 1998-2009 (2009). • (See also one-pagesummaries in - Norad Evaluation Department’s Annual Reports (2011,2010,2009)) http://www.norad.no/en/evaluation or contact us on post-eval@norad.no Side/Page

6. Findings across the evaluations 1) Thatthereareweaknesses in the analysis of the situation and of the conflict in both the planning and implementationphase – • Implications: • - limit relevance of the intervention/support/ • limit the evaluation’spossibility to saysomethingboutaboutrelevance (Afghanistan/ South Sudan) • reduces the likelihood of conflictsensitivityanalysisbeingconducted • More… Side/Page

7. Findings from the evaluations cont. • 2) Not enoughresourcesaresetaside to follow up and assess progress, during implementation (South Sudan, Afhganistan, Haiti and Sri Lanka). • At times, the staffing in embassieswere far from adequate(Afganistan- in particular, South Sudan, Haiti) Implications: • Not enoughresources to qualityassureprogrammes – aretheyon the right track (do we do whatwesayweshall do): (Afghanistan-50 % of scoolsnotadapted to girlsneeds (latrines/protectivewalls)). • Delays • The danger of aid not being relevant/not adapted to changingcontext or maybecontribute to aggraving the conflict(s) • Not enoughresources to monitor whetheraidbecomes a stake in the conflict Side/cccPage

8. Findings from evaluations cont. • 3) Too much emphasis from donors are put on harmonisation and coordination at the capital level in the partner country at the cost of sharing knowledge about local context, adapting activities to local conditions an presence in the field. • Example: South Sudan: Donor Coordination meetings did not revolve around sharing conflict analyses, and discussing how to coordinate aid to address local conflicts, but bigger diplomatic issues: referendum 2010. • Example: Afghanistan: the Norwegian funding to the ARTF remained remarkably consistent over the years despite important changes in the context Side/Page

9. Some key challenges to peacebuilding evaluations • The threat of violent conflict • The reality is oftencomplex, stakes arehigh, everythingbecomes political,– ‘all voices to be heard’? How to be perceived as impartial and balanced-key to the credibility of the evaluation? • Theory-poorfield • Evaluations can do harm (evaluatorsleave, othersstaybehind) Side/Page

10. How to deal with a challenging context… • How to conductconflictanalysis • How to conductconflict sensitive evaluations(do no harm) • How to analyse theories of change and their underlying assumptions • Surprises thatareexpected and thoseunexpected http://www.oecd.org/dac/evaluationofdevelopmentprogrammes/dcdndep/evaluatingconflictpreventionandpeacebuilding.htm. Side/Page

11. Findings from a (2012) meta review of evaluations of support to state building in fragile states: Few evaluations are concerned with: - explicating a theory of change; - constructing a credible counterfactual; - and making use of quantitative methods where possible. • Which leads to: • An inbuilt tendency to reproduce the conventional wisdom • instead of testing implicitly assumed causality chains • or exploring what the alternatives would have been. Side/Page

12. What do the evaluations of peacebuilding-field look at? • Peace buildingevaluationsconducted by many donors and mulitlateralorganisationsseem to have focusedonunderstanding and mapping the terrain and theirowninternalorganisation, • Theyareconcernedwith: • Coordinationbetween different actors, planning, • whether a conflictanalysis is used or not for programming (most often it is not), • types of interventions, (socio-economic, humanitarian, peacebuilding, governance) • conflictsensitivity, • Inputs, activities, and outputs Ratherthan • results,whatworksandwhatdoes not work(what do we base ourknowledgeon/ whichassumptions do we base the interventionson? Side/Page

13. Takk, thanks, asante, merci, شُكْرًا Side/Page

14. Findings from other evaluationssupport to state building 1) A mixed methods repeated survey (2007-2009) in north- eastern Afghanistan (2000 respondents in 80 villages ) commissioned by the German MFA found that development aid had a small positive effect on the populations’s attitudes towards foreign forces and the Afghan state. But, this effect dissappeared when the population experienced a deterioration in the security situation. Moreover, the small positive effect from aid on attitudes depended on the perception of aid as useful. The study also found that it is not the amount of projects that impact the afghan attitudes, but their perceived usefulness. Böhnke, J. R., J. Koehler, and Ch. Zuercher. 2010. “Assessing the Impact of Development Cooperation in North East Afghanistan 2005 – 2009. Final Report.” BMZ) Side/Page

15. State building continued 2) Statebuilding with emphasis on capacity building seem to be more relevant in post conflict situations, rather than in situations of ongoing conflict. This is based on findings from a Danish evaluation and a UNDP evaluation (2013) of statebuilding support to Somalia. The latter evaluation concludes: if objectives of strengthened governance systems shall be achieved, a minimum of stability is required. (“Evaluation of the Danish engagement in and around Somalia 2006-10”, 2011 http://www.netpublikationer.dk/um/11094/ “Evaluation of UNDP support to conflict-affected countries in the context of UN Peace Operations”, Draft final (not to be quoted), Evaluation Office, UNDP, Forthcoming January 2013.) Side/Page