What are Monitoring and Evaluation? How do we think about M&E in the context of the LAM Project?

1. What are Monitoring and Evaluation? How do we think about M&E in the context of the LAM Project?

2. Objectives of Monitoring & Evaluation Collect, analyze and use accurate and reliable information Brazil

3. Objectives of Monitoring & Evaluation • Improve program planning • Improve program performance • Improve program management • Determine outcomes of activities • Evaluate program impact on beneficiaries

4. Characteristics of Monitoring • Occurs throughout project on a regular basis • Provides early indication of progress, or a lack of progress • Provides periodic oversight of activity implementation

5. Characteristics of Monitoring • Can be tasked to any staff, not a specific evaluation staff person • Informs a project about strengths & weaknesses: do we refine our strategy?

6. Characteristics of Monitoring • Determines whether activities are being performed correctly • To what extent are planned intervention activities being realized? • What services are provided, to whom, when, how often, for how long and in what context?

7. Is Monitoring Supervision? • We monitor an intervention • We supervise an individual

8. Characteristics of Evaluation • Time-bound • Systematic and objective • Assess performance and impact • Carried out selectively – costly & time consuming • Process & Impact • Process: organization & management • Impact: effects on beneficiaries

9. Types of Evaluations • Baseline Surveys: provides project with information about the target population at beginning of project intervention • Knowledge about breastfeeding, complementary feeding, maternal nutrition, and LAM (BCM/LAM) • Practices related to BCM/LAM • Cultural beliefs related to BCM/LAM • Household-based in the community • Large, representative sample

10. Types of Evaluations • Endline Surveys: provides project with information about the target population at the end of project intervention • The endline survey is compared to the baseline and like the baseline it is: - Household-based in the community - Large, representative sample

11. Types of Evaluations Endline Surveys: The endline asks the following questions: • Did we meet our objectives? • Did we have an impact: is there a measurable difference in people’s knowledge & practice? • Where were we strong and where were we weak? • What problems remain to be addressed?

12. Types of Evaluations • Performance Monitoring: measures knowledge & skill levels of individuals trained in the field • Results are used to refine training strategy & define future training needs • Results are used for better supervisory visits

13. Types of Evaluations Performance Monitoring: • Ideally, performance monitoring evaluations are not necessary • An effective supervisory system would include on-going evaluation of field worker’s knowledge & skills (counseling, methodologies) following training

14. Monitoring & Evaluation Indicators How do we track & measure progress over time? Guatemala

15. What is an Indicator? • A variable • That measures • One aspect of a program

16. What is an Indicator? • A measure of whether you are heading in the right direction • A measure of what your program has achieved • A measure of the behavior you’re monitoring

17. An appropriate set of indicators will include at least one for each significant element of the program

18. Characteristics of Indicators • Quantitatively or qualitatively measurable • Relevant to the goals they represent • Objectively verifiable & reliable • Meet international professional standards • Understandable & appreciated by project participants and other stakeholders • Clarity on the appropriate levels: clinic-specific, country wide

19. Characteristics of Indicators • Clarity on the appropriate levels • Is the indicator clinic-specific? • Is the indicator region-specific? • Is the indicator country wide?

20. Developing Indicators • Establish during initial program planning • Reflect overall objectives • Link to behaviors • Include with baseline

21. How To Measure Indicators? • Knowledge, practice and coverage surveys (KPC) • Health Facility Records • Health Information Systems • Formative research to identify behaviors & indicators

22. LAM Indicators 1. Exclusive Breastfeeding Rate, Infants 0 - < 6 months Guatemala

23. LAM Indicators Exclusive Breastfeeding Rate, Infants 0 - < 6 months • Age range for LAM to be applied • Eligible LAM users • Optimal infant feeding message • Includes ALL infants in the age range • Can also be written 0 – 5 months

24. Exclusive Breastfeeding Rate, Infants 0 - < 6 months • WHO indicator using 24 hour recall • Reflects women eligible to use LAM • 1995: 46.1% • 1998 – 1999: 38.8% Decrease of 7.3% Source: Guatemala DHS Reports

25. LAM Indicators 2.LAM Rate: Proportion of eligible women (infants less than 6 months), who choose LAM as a method of birth spacing Altiplano, Bolivia

26. LAM Rate • Formula: Current LAM users divided by all women with infants 0 < 6 months • Accuracy: A true estimate of LAM use by eligible women • Data Source: Service statistics (MIS) & Household-level survey (DHS)

27. LAM Indicators 3. LAM User Rate: Proportion of all women of reproductive age who use a modern contraceptive method (few eligible to use LAM) who choose LAM as a method of birth spacing Llano, Bolivia

28. LAM User Rate • Formula: Current LAM users divided by all women of reproductive age who use a modern method of family planning • Accuracy: Underestimates actual LAM use by eligible women • Data Source: Service statistics (MIS) & DHS

29. LAM Indicators 4. Appropriate LAM Rate: Proportion of women who give birth in a given period of time who consciously & deliberately accept LAM as a birth spacing method and meet the 3 LAM criteria Guatemala

30. Appropriate LAM Rate • Formula: Number of women who use LAM as a birth spacing method and meet the 3 LAM criteria divided by the total number of women with infants 0 < 6 months

31. LAM Indicators 5. Model LAM Rate: Proportion of women who give birth in a given period of time who consciously & deliberately accept LAM as a birth spacing method, meet the 3 LAM criteria, and know the 3 LAM criteria Jordan

32. Model LAM Rate • Formula: Number of women who use LAM as a birth spacing method, meet the 3 LAM criteria, and know the 3 LAM criteria divided by the total number of women with infants 0 < 6 months

33. Denominators • Exclusive Breastfeeding Rate: • Infants 0 < 6 months • LAM Rate: • Women with infants 0 < 6 months • LAM User Rate: • Women of reproductive age using a modern contraceptive method • Appropriate LAM Rate: • Women with infants 0 < 6 months • Model LAM Rate: • Women with infants 0 < 6 months

34. LAM Monitoring & Evaluation Strategies • Integration of LAM as a contraceptive method into the national MIS • LAM acceptors integrated into national MOH/FP data

35. Lessons Learned • Increase and sustain LAM Users • Optimal breastfeeding practices • Integrated & supportive environment

36. Lessons Learned • Data Collection • Monitor clinics for accurate data collection • Absolute LAM numbers do not reflect LAM success – need proportion • Additional information - Age of infant - Family Planning Transition information - Length of time using LAM

37. Lessons Learned • Integrate the concept of Monitoring • Utilize results of supervisory checklist for program management

38. Next Steps • Review monthly and quarterly data for changing LAM trends • Review MIS and supervision data on a regular basis • Conduct regular review meetings and analysis for program management