PADEV

1. PADEV PARTICIPATORY ASSESSMENT OF DEVELOPMENT a new methodology for impact assessment

2. PADEV Case Presentation for the conference Evaluation Revisited By Fred Zaal and Wouter Rijneveld Utrecht, May 20, 2010

3. Contents • Background of the case • Description of the case • Methodology • Illustrative findings • Values, Quality, Complexity • Improvements foreseen

4. Background of the case Perceiving shortcomings of ‘impact evaluations’ • Not really about impact (rather outcome – output) , too short term • Lack of attention for context • External factors • Other interventions • Focus on predetermined (by whom?) results • If participative, then ‘dependency bias’

5. Background of the case Longing for • True participation and ownership • True holism, but not vagueness • Getting rid of the ‘dependency bias’ And seeing preliminary work of Ton Dietz in Kenya…

6. Description of the case ICCO, Prisma and Woord en Daad (Dutch, 2 NGO’s 1 network) joined hands with • University of Amsterdam (NL) • University of Development Studies (Ghana) • Expertise pour le Développement au Sahel (Burkina Faso)

7. Usualperspective backdonor INGO • Activities • Results • Outputs • Outcomes • Impact • PM&E NGO Project a in community x

8. Other backdonors Other INGO’s NGO 2 NGO 3 • Government • Local • State • National Project c Project c Project b Project b Project a Project a Companies (e.g. telecom) Natural Physical Economic Human Social Political Cultural Private initiatives backdonor INGO NGO 1 Project c Project b Project a Community x Community x Very poor – poor – average – rich – very rich Local influences National influences Global influences

9. Methodology Projects Actors History History Community x Very poor – poor – average – rich – very rich History Changes in context

10. 3 day workshops • 60 people from area of 20,000 • Subgroups: men, women, old, young(officials and project staff separate group) • Individual life history questionnaire data about participants, parents, siblings, children (total 600 persons per workshop) • 4 rounds of 3 workshops each(3 x 3 workshops concluded)

16. Contents of workshop Toolbox of exercises: • Context: shocks, trends (6 capitals), wealth classes • Interventions: inventory (by actor, sector, years) + valuation • 5 best / worst interventions: in depth exercises: • Shift in perception over time • Effects on wealth classes and capitals • Relation between trends and interventions (cause, mitigation)

17. Illustrativefindings • Incredibly detailed historical profile + descriptions of wealth categories

18. Numbers of Projects

19. Best / Worst projects

20. Impact on wealth classes

21. Impact on wealth classes

22. Values and Principles • Principles: • Evaluation = “valuing” something (diversified) values of the people • Evaluation (esp. at impact level) must be done in context, relative to other factors and actors. • Values: • Emerge from participants

23. Quality Standards • Validity (do you measure what you want to measure?) • Impact = long term • Inter-subjectivity (negotiated consensus) instead of assumed objectivity • Reliability / replicability(are findings stable?) • Convergence of type of results after 9 workshops • Additional trials at different scale / different target groups (some still in process) • Meticulous reporting, External Reference Group, AMAP Feed back

24. Complexity… … is the essence of this method Multiple relations between shocks, trends (per capital, function of time), interventions (per type of actor / sector, function of time) and effects on people in wealth classes, capitals (function of time)  Insight in some of these relations

25. Complexity

26. Improvements / trade-offs • Triangulation • With external data (e.g. on trends) • With the Partner agency’s knowledge and monitoring system • Selection of participants • ‘opinion leaders’ (officials are separated) • Trade off between ‘randomization’ and needs for analyses of the groups