Observe and evaluate change : an overview of change- oriented approaches

1. Observe and evaluate change : an overview of change-orientedapproaches 10 November 2017 _ InnoVET

2. F3E / Prisme • F3E, a multi-stakeholder network • Mission: Evaluate, Enlight, Exchange • PRISME: research – action programme to experiment Change-orientedapproaches

3. Training objectives • Understanding change-orientedapproaches(COA) : • Whytheycanbeuseful; • How theycanbeused; • Gainingmethodologicalhints : • Test COA to evaluate how InnoVETcancontribute (or not!) to change for you • Experimentdrawing« change paths » with a given vision Short time: choiceshad to be made, sorry for the Franglish… ASK QUESTIONS IF NEEDED

4. Programme (in very short) 1) Morning: Discussexistingtools, a video on COA and a 1st experimentwith the InnoVETprogramme evaluationtools 2)Afternoon : Some insights on COA and a practical exercice on drawing « change paths »

5. 1. Explore existingtools In 2 Groups: 10’ • Evaluation Googleform: whatdidyouthink? (+/-) • Fiche capi: whatdidyouthink? (+/-)

6. Let’s do thingsdifferently! hatis the scope of potential change?

7. Exercice: Per organisations: try to answerthese questions: • Whatchangedrecently? When/who/what • Why? (explain, characterize,…) • So what? Whatdoesthis tell us about ourproject ? • Whatnext? Decisions to be made?

8. Change Flower

9. Whatdidyouthinkof this short exercice?

10. Programme (in very short) 1) Morning: Discussexistingtools, a video on COA and a 1st experimentwith the InnoVETprogramme evaluationtools 2)Afternoon : Some insights on COA and a practical exercice on drawing « change paths »

11. What are Change-orientedappraoches?

12. Gettingready to evaluate change … • Monitoring and evaluation / measuring impact : • Usuallystartswithanalysingactivities, and see how theycontributed to predefined objectives • Everythingispredefined, includingindicators • Usuallywith quantitative data more than qualitative. • Whenwework in verycomplex settings, and we are working on untangible aspects: whatactuallyreallycountsiswhat CHANGED and how wefacilitate CHANGE

13. Origins of the Prisme • How to evaluate qualitative change? • Variousinspiring techniques: • Theory of change: ToC • Outcomemapping • Outcomeharvesting • Most significant change: MSC • 2 main approaches • DEDUCTIVE (TOC) • INDUCTIVE (OH/MSC) • PRISME : a happy mixture of all techniques! Pour en savoir plus : www.f3e.asso.fr/ressource/45

14. Key principles COA versus ROA (Resultsorientedapporaches) • Start with a positive vision (avoidstartingwith the problems) • Focus onstakeholders(not actions) • Think of the stages needed to reachour vision : progression • Facilitatestakeholders in becoming « authors » of theirown change (participation/ownership)

15. Takecomplexity and stakeholdersintoaccount

16. PROGRESSION Long term objectives/Outcomes ? Activities/outputs

17. Collective M&E

18. Most important are principles, thentoolscanbeadapted

19. 1. Contexte / Context 2. Vision 3. Change paths 4. Activités / Activities 5. Suivi-évaluation / M&E

20. Questions ?

21. 1. Contexte / Context 2. Vision 3. Change paths 4. Activités / Activities 5. Suivi-évaluation / M&E

22. Example of tools : Timeline Steps 2011 2017 Expectedresult: a shared « historical »analysis Factors / stakeolders

23. Tool: SWOT FFOM Analysis Objective; analyse collectively;  SWOT / FFOM.

24. Panorama des acteurs Stakeholders: generalmap

25. Stakeholderanalysis (examplelinked to the vision)

26. Change stakeholdersmapping Power to influence change Veryinluent/i verymportant + Not favorable Influent Favorable Influent + – Interest for change Blockers/resistant to change Driver / Favorable to change Not favorable Not influent Favorable Not influent – Not very influent / not very important

27. Identifystakeholdersthatcan drive potential change 4 formes de pouvoir / 4 forms of power : • le pouvoir sur (power over) • le pouvoir de (powerto) • le pouvoir avec  (powerwith) • le pouvoir intérieur (powerwithin)

28. Analyse the context Questions to raise? • What are the constraints and opportunitieswithinourenvironment,? • Who are the key stakeholders ? Learnings Prisme ? • Identifing new stakeholders • Betterunderstanding of the relationshipsbetweenstakeholders • Difficult to analyse stakeholdersrelationshipswithinprojectsdelt by networks • Very important stepthatneeds time: willform the basis of change-oriented planning • Doingitcollectivelywithstakeholdersalreadyinvolved to give a sense of responsibility

29. 1. Contexte / Context 2. Vision 3. Change paths 4. Activités / Activities 5. Suivi-évaluation / M&E

30. HOW? • Writeindividually, then in small groups, then in a bigger group • Identify a small group of people to reformulate When/who/what • You canalsodraw a vision!

31. Exemple de vision  • En 2025, Cendajuru est une commune riante et attractive. Elle bénéficie d’une agriculture prospère et durable, qui a permis d’améliorer les conditions de vie de l’ensemble de ses habitants. Elle est dotée d’un niveau d’infrastructures et de services satisfaisant. Toutes les terres ont été enregistrées, ce qui a permis de créer un climat social apaisé. Les habitants et acteurs de la commune ont renforcé leurs capacités et une véritable élite intellectuelle s’est développée, qui s’investit pour le bien commun de Cendajuru.

32. Define a long term vision • Questions ? • To whatwillwecontribute in 5-10-15 years time? • Identify the most important areas covered by our vision? Learningsfrom the Prisme ? • Powerful/mobilizing • Takes time: thinkwellyourmethodology! • Helpsspecifyevenfurther key stakeholders • Depends on the quality of the analysis of yourcontext

34. 1. Contexte / Context 2. Vision 3. Chemins /Change paths 4. Activités / Activities 5. Suivi-évaluation / M&E

35. 1 stakeholder = 1 main challenge/ big change = 1 or severalchange path(s)

36. Change paths: how to drawthem? Change Change Time line : Identifyintermediary changes and activitiesthatcancontribute to this change Activity Activity • Tips/questions to identifyintermediary changes: • « To acheivethisbig change / challenge, what do weneed to getdone? » I need… Weneed… We have to… • Is thisreally a change or an activity? • «Behindthisactivitywhat do wewant to reachexactly?»

37. Stakeholders and main change paths: a visualexample A renforcé ses capacités Offre des services (SFC) et infrastructures Coordonne l’action des différents acteurs A passé le relai à la commune Est intégré dans la commune / pérenne A renforcé ses capacités Joue rôle de coordination de la société civile Administration communale Est une référence nationale A contribué à un modèle national ACORD Service foncier communal Vision A accomp. intégration agro-écologie et foncier Comités de reconnaissance collinaires Cadres, élite intellectuelle Exploitants, ménages (hom., fem.) Ont convaincu toute la communauté / enregistrer Autres intervenants du SAF Ont enregistré toutes leurs parcelles Sont fiers du service rendu à la communauté Ont dvpé production, cultures variées + gestion Femmes & hommes sont impliqués / gestion terres

38. Example of a detailed change path for 1 stakeholder

39. Exercice : drawyour change pathslooking back at your main challenge

40. Whatdidyouthinkof this short exercice?

41. Design change paths Questions  ? • What are the main challenges? • What are the main stepstowardseach of these challenges? Learningsfrom the Prisme ? • Key step in the change-oriented planning process: alsoless « natural », unsusual • Bringsdoubts for stakeholders; difficult to distinguishactivity, result,s, outcomes, outputs, and CHANGE. • Attention to progression is not necessarilythinking in a linearway (possible to drawwithparalells, curves) • Difficult to facilitate!

42. 1. Contexte / Context 2. Vision 3. Change paths 4. Activités / Activities 5. Suivi-évaluation / M&E

43. Exemple change path / activities

44. Defineactivities and strategies to facilitate change • Questions ? • Whatstrategies / activities /behaviours to facilitatethis change? • Explicit hypothesisbehind the activities: why do youthinktheycancontribute • to change? Learningsfrom the Prisme? • Enables a more strategic design… • … if youavoid the activity « shopping list » • Giveseven a sense of « coresponsibility » on change paths, especially for territorial dynamics

45. 1. Contexte / Context 2. Vision 3. Change paths 4. Activités / Activities 5. Suivi-évaluation / M&E

46. Data collection / Analysis: an example Unexpectedchange?

47. Monitoring and evaluating Change Questions  ? • How canwe monitor ourevolution on change paths? • What are the key learnings for ourproject? Learningsfrom the Prisme ? • You need to daremaking the M&E system evolve, and makeit « light » / realistic • Concentrate on change pathsthat are really important • Rolesneed to bewelldefined: whocollects/whoanlyses? • It isusuallybetterwhencollecting to leavedoors open: MSC technique canbeuseful

48. Questions ?TESTIMONY UNMFREO

49. Ressources Toolkik: videos, PPT, methodological guides www.f3e.asso.fr/article/1671/boite-a-outils

50. Possible quantitative indicatorsto follow up: • XXX% changed their vision on the programmeon the programme main theme ? • XXX% changed their vision of exploring the context • XXX% changed their vision on the methods and tools • XXX% changed their approaches of collaborative work in the group • XXX% changed their vision on methodological approaches they use in their organisations • Extra: (level of satisfaction) At least XXXX number of smileys ? OTHER?