Monitoring & Evaluation (M&E)

1. Monitoring & Evaluation (M&E)

2. Overview of Day 3 • M&E What and Why • Data sources and tools • Baseline and Targets • M&E How – Tools Presentation/Q&A • Lunch • M&E Case study: African Platform for Social Protection/Q&A • Monitoring Plan Group Working Session/Q&A • Working session – Developing M&E plans • Tea break • Working session – Developing M&E plans continued • Short presentations and feedback • End of day real time evaluation

3. M&E What • Monitoring is the process for checking project implementation – resource-use, that activities and outputs are on track, that results are in line with those that were planned and looking out for results that were unplanned. • Evaluation is the systematic assessment of an ongoing or completed project, its design, implementation and results. In evaluation, what has happened is important – both planned and unplanned results, implementation - and external factors (reasons/ causes). Evaluation also refers to the process of determining the worth or significance of an activity or project. • In practice, these distinctions between monitoring and evaluation have become blurred, but there are clear differences …

5. M&E Why • Provide key stakeholders with information and knowledge about the project. • Produce reliable data for the purposes of reporting on outputs and outcomes. • Fosters a culture of continuous improvement to build on quality or adapt the project to be responsive to the needs of target beneficiaries. • Learn about the short, mid and longer-term contribution of the project to high level objectives to inform new project decisions and future strategy. • Advocacy for resources and new partnerships.

6. Participatory M&E (PM&E) • The Foundation is dedicated to strengthening people’s participation in all aspects of public dialogue, to act together and learn from each other to build democratic societies. • With growing emphasis on participatory approaches towards development, there has been recognition that M&E should also be participatory. • Conventionally M&E involves an outside expert coming in and measuring performance against pre-set indicators - now replaced by PM&E where primary stakeholders are active participants— not just sources of information. • Stakeholders evaluate, outsiders facilitate. • Advantages • Builds stakeholder capacity for analysis and problem-solving and learning of all involved. • Increases the reliability of information and opportunity to receive ideas for corrective actions. • Strengthens ownership regarding successful outcomes of planned initiatives.

7. Types of Data

8. Data Sources and Tools

9. Challenges in Data Collection • Badly planned Logframe: Indicators that are not matched to a means of verification (MOVs), Badly chosen indicators and MOVs. • Validating data: trustworthy secondary databases. • Securing feedback from less visible or vocal stakeholders: good sampling techniques, especially for participation of less-heard voices i.e women and girls. • Consent and Information: Working with children and other vulnerable groups (HIV/refugees, disabilities).Data protection and seek consent before circulating images or video on social media.

10. Baseline • Baselines are quantitative and/or qualitative data collected (on a project or an activity) to establish the pre-project conditions against which future changes can be measured. • Establishing priority areas/planningfor a project, especially true when a project has several objectives - disaggregate your baseline where possible. • Attribution: Without a baseline, it is harder to identify the outcomes/impact of a project, especially if other organisations have implemented projects in a similar area. • Targets: It is difficult to establish a reasonable performance target without a baseline. • Where feasible and relevant develop a baseline for outcome indicators. • Baseline research should not be confused with a rapid assessment or a situation analysis, although if this is done close to project start-up, data from rapid assessments or situation analysis may consist of valid information for baselines

11. Game • Scenario 1: The context for a project supporting women to access maternal health services has changed over the past year. Therefore, the project decides to do a to determine what are the causes of increased maternal mortality in the local community; what are its effects on different groups; how do different people perceive it; what is the political/duty bearers context; what different groups and institutions are stakeholders; what are the links between different organisations; what policies are concerned and how are they implemented. Situational analysis • Scenario 2: Monsoon rains have severely affected a project supporting quality education to children aged 4-11. The project decides to carry out a to find out more about what has happened, which children and how many of them are affected. rapid assessment • Scenario 3: A new 3-year project is starting and as one of the first activities of the project’s implementation is carrying out a to establish what are the values of project outcome indicators in this first year. This information will then be used to set targets against which future change in the target population will be measured. baseline study

12. No Baseline • Natural disaster or civil unrest may not have made it possible to determine baseline conditions before implementation begins . • Lack of awareness on the value of baseline data for monitoring purposes. • Financial constraints. • It is generally not possible to conduct a rigorous evaluation without solid baseline data. However, the non-existence of baseline data does not have to spell the doom and gloom. • Using previous studies as a baseline: Several studies are conducted by different agencies including national surveys and sectorial surveys. • Partners: In some cases, it could necessary to involve other organisationsin the baseline survey. This is especially viable if “similar” projects share a starting timeline and share a target group, most often by projects sharing a donor. This normally saves costs an increases confidence in the baseline results.

13. Targets • Definition:Targets represent commitments to achievements in concrete terms. It is the desired end point for each indicator which can be achieved as an output/outcomes. • Once a program is underway, targets serve as the guideposts for monitoring whether progress is being made on schedule and at the levels originally envisioned. • Rationale: A target should be: • Realistic: Given resources and capacity in the project as well as the current baseline, • Achievable: Within the time period available in the project • Measurable: Such that you know you reached it and can provide evidence • Disaggregate: Providing data on how the outcome impacted where relevant by women and girls (gender)/age (young, elderly), ethnicity, sexuality, disability. • Develop qualitative or quantitative targets for outputs and outcomes. Ensure they relate to a means of verification and your baseline.

14. Baseline and Targets - Disaggregation • Provide nuanced information that may not be obvious in the aggregate. • The Foundation learning on excluded groups- (women, indigenous communities, religious groups) and other less heard voices…this forms a key aspect of understanding how PG affects groups differently. • Where appropriate disaggregate the data for groups based on gender/age (young/elderly), ethnicity, sexual orientation, disability, religion etc. • By disaggregating baselines the project can get an understanding on how it can address the needs for various groups. • By disaggregating targets, programme inputs can be directed to reach a particular target group.

15. Storytelling for Gender Equality Project • Develop qualitative or quantitative targets for outputs and outcomes. Ensure they relate to a means of verification and your baseline.

16. Any Questions

17. M&E How - Tools

18. Gender – Intersectionality • Intersectionality- gender inequalities intersect with other aspects of oppression, resulting in unique constellations that include not just gender but race, sexuality, ability age, social class, caste and various others. • An analytical tool for studying, understanding and responding to the way in which gender intersects with other identities and how these intersections contribute to unique experiences of oppression and privilege. • Asking gender intersectionality questions delves deeper into cross-cutting factors. • It is used in research, analysis and planning. Examples of gender-based questions include what the Foundation has trialled with partners: • What forms of identity are critical organising principles for this community/region (beyond gender, consider race, ethnicity, religion, citizenship, age, caste, ability)? • Who are the most marginalised women, girls, men and boys in the community and why? • Which laws, policies and organisational practices limit opportunities of different groups?

19. Advocacy- Intense Period Debrief • A tool for small groups to evaluate the impact of advocacy activities. • Data capturing directly after a policy window or intense period of action. • Fills in the gaps in information-gathering. • An opportunity for learning. • Useful feedback for future use.

20. Case study • Coalition for Comprehensive Immigration Reform (CCIR)- a bipartisan coalition of diverse groups and national organisations advocating for comprehensive immigration reform. • CCIR’s lobbying and grassroots efforts after advocating for immigration policy reform were analysed in a debrief. • Intense Period Debrief tool was used to follow the highs and lows of the policy advocacy cycle. The following information was captured: • the public mood and political context of the opportunity window; • what happened, and how the campaign members responded to events; • what strategies they followed; • their perspective on the outcome(s) of the period’s activities; • and how they would change their strategies going forward based on what they learned during the period.

21. Stories of Change • A case study method to determine a project’s pathways of success. • Useful to investigate impact through first-hand accounts and analyse how activities cause impact. • Used to communicate the benefits of a project. • Used in retrospective evaluation or review.

22. Digital Stories-Migration Asylum Project • M.A.P. is a Grants funded project currently using digital stories as part of a University of York cross jurisdictional working group – to understand legal engagement and improve access to justice for refugees. • An oral narrative accompanied by a series of visual elements that helps tell the story. • Humanises experiences and can be shared with a wide audience and across multiple platforms. • Digital story captures experiences of legal encounters between lawyers and refugees narrated by refugees themselves. M.A.P communicated the benefits of a project through the following process: • The list • Each refugee (client) is spoken to individually and informed consent is drawn. If needed, a translator is called. • Speaking to each client individually • Retrieving content • Voice recording and filming • Storyboarding • The Final Product Migration Asylum Project Digital Story

23. Most Significant Change • A participatory method to determine a project’s impact through the perspective of different stakeholders. • Similar to a Stories of Change, however, it is used to determine the most significant impact of a project. • It is used in retrospective evaluation or review.

24. M&E Tools Working Session & Feedback

25. Any Questions

26. We expect you to create a Monitoring Plan.

27. Developing Monitoring Plans • Design tools • surveys – in collaboration with local teams • sampling techniques where relevant • assess if tools can be used on marginalised groups • Plan frequency and timeline for data collection • Pair up with other activities where feasible • Assess local feasibility for data collection: weather, festivals, availability of staff and target group especially women • Assess skills and capacity of • Project team to collect and analyze data • Analysis software and cost for purchase if needed • Consultant fees for evaluation if planned • Budget • Costing data collection and analysis in project logframe • 10% of budget on monitoring, assessment and learning in Commonwealth Foundation projects

28. Group Working Sessions

29. Individual Working Session: Create a Monitoring Plan • Using the template for your project log frame: • One Outcome Indicator

30. Presentations and Feedback

31. Thank you Anita Nzeh Senior Programme Officer Knowledge, Learning and Communications (KLC) a.nzeh@commonwealth.int