Monitoring and Evaluation: Strategy and Framework Proposal

1. Monitoring and Evaluation:Strategy and Framework Proposal Ekin Birol on behalf of H+ MLETask Force

2. The need • BMGF meeting last year • Confusion between MLEand the research impact team does • Increasing requirements for reporting indicators (USAID, DfID, now BMGF…) • Apt review • Confusion and differences in what indicators are and should be • Unavailability of readily available, systematic, uniform, timely data

3. H+ MLETask Force • Ignatius Abaijuka • Dorene Asare-Marfo • Ekin Birol • Michael Diressie • Abdoul Murekezi • Adewale Oparinde • Manfred Zeller • With input from • Olyn Panlilio • Amy Saltzman

4. One week workshop • Met March 18-22 • Discuss why we need to do MLEat H+ • Develop a “mock” MLEFramework for delivery (Zambia and Rwanda PDDPs) • Discuss how we can implement this framework • Here is what we came up with….

5. What is MLE? • Monitoring is the systematic and routine collection of information from projects and programmes for three main purposes: • To learn from experiences to improve practices and activities in the future; • To have internal and external accountability of the resources used and the results obtained; • To take informed decisions on the future of the initiative;

6. What is MLE? • Monitoring is a periodically recurring task already beginning in the planning stage of a project or programme. • Monitoring allows results, processes and experiences to be documented and used as a basis to steer decision-making and learning processes. • Monitoring is checking progress against plans. • The data acquired through monitoring is used for evaluation.

7. What is MLE? • Evaluation is assessing, as systematically and objectively as possible, a completed project or programme (or a phase of an ongoing project or programme that has been completed). • Evaluations appraise data and information that inform strategic decisions, thus improving the project or programme in the future.

8. What is MLE? • Evaluations should help to draw conclusions about four main aspects of the intervention: • Effectiveness • Efficiency • Impact • Sustainability • Information gathered in relation to these aspects during the monitoring process provides the basis for the evaluative analysis.

9. Why we need to do MLEat H+ • We need to keep systematic, comprehensive and uniform records for several reasons • Accountability • Auditing • Reporting • Recording • Planning • Learning • Improving • Expanding

10. Keep records of indicators • Input => Activity (process) => Output => Outcome => Impact • All indicators (except impact) must be SMART Simple Measurable Achievable Realistic Timebound • SMART indicator rule helps to rule out theoretically correct but hard to measure indicators SMART indicator rule helps to rule out theoretically correct but hard to measure indicators

11. Examples of H+ MLEIndicators

12. How to measure these indicators? E M • Impact indicators: • To be measured by the impact team through effectiveness studies and modelling • Outcome indicators: • Should be measured jointly by the impact and country teams • Input – Process- Output indicators • Should be measured by the country teams • Example Zambia MLEplan ME plan Zambia.xlsx

13. How to measure these indicators? • Need commitment by the PMT • Close collaboration between (i) country and impact teams; (ii) finance and MLE • Need to agree on overall and country specific SMART indicators • Disaggregation, baseline, other similar projects.. • Need funds and staff • MLEStaff in country offices - progress • MLEStaff in HQ managing/leading the MLEsystem • 5% of the total H+ budget

14. How to measure these indicators? • Need to develop the MLEsystem and other related systems • MLESystem includes: • Logical framework matrix - what we intend to do and how, key assumptions and how outputs and outcomes will be monitored and evaluated. • MLEplan – linking objectives to indicators to data • Various data collection tools, database templates, reporting templates • Other related systems: • Cost accounting system – linking costs with activities • Ideally MIS – Management Information System

15. How to build MLESystem at H+? • Need an overall MLEManager • Directly reporting to the Director (e.g., IEG at WB) • Evaluate all documents – PDDP, budgets, workplans, talk with all managers • If needed work with a consultancy firm to develop the generic MLEsystem • Develop a web-based MLEsystem (software on intranet for uniform/systematic reporting) • Work closely with country MLEspecialists and all managers and finance to develop and implement country and programme level MLEsystems • Lead, monitor and evaluate the MLEat H+

16. Counting number of adopters – A SUGGESTION

17. How to count no of adopters • First need to keep record of recipients • Name, location, phone number, variety, quantity, gender • Output indicator • Country team’s job • Nigeria, Uganda (NGOs) • India, Zambia (seed companies, agri-input shops) • Rwanda (several delivery methods and partners) • Follow up with a representative sample of recipients • Ask adoption, consumption, sale, intent, diffusion, gender, area • Outcome indicator • Every season? Impact and country teams do together