29th-2 nd /03/2009 Conference AfrEA - NONIE - 3ie Cairo , Egy pt

1. 29th-2nd/03/2009 ConferenceAfrEA - NONIE - 3ie Cairo, Egypt Olivia Nakayiza & Luc Vanhoorickx – Save the Children in Uganda

2. Programme Overview Save the Children in Uganda (SCiUG) is a unification of Save the Children Norway (Managing Member), Save the Children Denmark, Save the Children USA, Save the Children UK, Save the Children Sweden and Save the Children Italy • In 2008: • SCiUG programmes in over 26 districts in 4 regions of Uganda • SCiUG works with 59 partners including local NGOs, CBOs and local government. • Total 313 staff as per December 2008 • Reached 213,051 children (116,137 girls & 96,915 boys)

3. Strategic Choices In the year of unification, Save the Children in Uganda (SCiUG) developed a strategic framework that builds both on the strengths of participating members as well as on emerging analyses of the child rights situation that calls for more integrated programming SCiUG has also continued to scale up its emergency response capacity to deal with emergency contexts, focusing primarily on emergency education and child protection

4. Community Based Family Planning Program Background Increased voluntary use of key family planning services and RH behaviors among women and men of RA • A 4 year USAID/FLEXIBLE Fund Community Based FP/RH- Luwero, Nakaseke and Nakasongola Districts in Uganda • Women of Reproductive age (15-49) 68,826, Married WRA 33,036, Men 68,423. Total # of beneficiaries 170,285 • Partnerships: District, MoH, etc

5. FP/RH Strategies and Intermediate results strategic objective : “Increased and sustainable use of key family planning services and RH behaviors among women and men of reproductive age”. INTERMEDIATE RESULTS IR-1: Increased knowledge, interest and use of FP/RH services IR-2: Improved quality of FP service delivery by providers at the facility and community level IR-3: Increased access of communities to FP services and information IR-4: Improved social and policy environment for FP/RH services and behaviors • Partnership Defined Quality (PDQ) process…… • Behavior Change Communication, … • community-based services to increase access to FP methods ,... • Build capacity of district staff, …

6. Time Impacts Longer Term Outcomes Medium Term Inputs & Outputs Shorter Term Results FrameworksWhere is results monitoring? Goal RESULTS MONITORING: Strategic Objective Check change in knowledge, behavior, status IR 1 IR 2 IR 3 IR 4 Strategies Activities

7. Monitoring & Evaluation => LQAS Higher level monitoring  Measure outcome indicators : compare with baseline & final targets From the Monitoring we knew that we were generally carrying out the activities as planned, but wanted to Monitor&Evaluate whether it all had the desired effect/result/outcome/impact  Effects of Behavior Change campaign: … behaviors => information needed = population/beneficiary based => sampling needed ! LQAS = Lot Quality Assurance Sampling • “determination of the quality of a lot by sampling” Lot = sub county from a district  small sample size, typically 19 per area  comparison between areas (lots) possible

8. What is LQAS? • A sampling method that: • Can be used locally, at the level of a “supervision area,” to identify priority areas or indicators that are not reaching average coverage or an established benchmark • Can provide an accurate measure of coverage or health system quality at a more aggregate level (e.g. program area)

9. LQAS: dichotomic - 19 Lot Quality Assurance Sampling • in general small sample of 19 provides an acceptable level of error for making management decisions; at least 92% of the time, it identifies whether a coverage benchmark has been reached or whether a supervision is substantially below the average coverage of a program area. • Samples largerthan 19 have practically the samestatisticalprecision as 19. They do not result in better information, and theycost more For example, what LQAS can: just by sampling 19 women in a targeted population, at least 92% of the time LQAS will determine correctly whether yes or no these women have adopted the family planning method.  LQAS cannot be used for coverage estimates in lot / district!  Only if it’s above/below target  Decision Rule

10. Some indicators that were measured • Contraceptive Use (CU) or Contraceptive Prevalence Rate (also known as Met Need)Percentage of women of reproductive age (WRA) 15-49 who are married or in union using (or whose partner is using) a modern method of family planning • Percentage of women of reproductive age (WRA) currently married or in union who are fecund (not pregnant or unsure if they are pregnant and not sterilized) who desire to have no more children or postpone childbearing, and who are not currently using a method of family planning (Unmet Need for Family Planning) • Percent of Demand (Met need + Unmet need) satisfied • Knowledge about family planning methods: FP METHODS THAT WERE MEASURED • Depo-Provera • Pills • Condoms • Norplant and Tubal Ligation

11. LQAS in Family Planning project As the mid-term evaluation of the Family Planning project Effects/outcomes of project interventions:… behaviors % of women of reproductive age who report having access to FP services % of women of reproductive age who were counseled about the birth spacing • Sampling: Systematic Random Sampling from … • 5 lots were chosen for different districts • 2-step random sampling procedure. Communities were selected using systematic random sampling. HH selected using classical random sampling Data collection tools: Adapted from FlexFund survey guidelines Data analysis: Standard LQAS tables in Excel worksheets for analysis & graphs

12. Process • 5 “supervision areas” • Random samples of 19 from monitoring records of women of reproductive age (WRA) • Interchanged extension workers for data collection • Minimal data-entry where PDA were used • Instant results • Fast feedback

13. LQAS Summary Tabulation

14. FP LQAS OUTCOMES

15. Average Coverage

16. After the quantitative survey… • LQAS gives the numbers (it’s quantitative) • Identifies the problem • How big the problem is • Where the problem is • BUT NOT why we have the problem… • Hence the need for qualitativeresearch /discussions to • Explain quantitative results • Identify the reasons why • Provide recommendations on program design and strategies

17. Lessons Learned (1) • Internal interviewers were used but swapped to reduce biasQuantitative LQAS survey replicated yearly • Although some statistical analysis is not possible with LQAS, indicators can be analyzed further to reveal additional informatione.g. analyze which ‘part’ of knowledge / behavior adopted => focusing messages

18. Lessons Learned (2) • Results discussed w/ program staff during participatory feedback meetings: discussions & decision making w/officers + review meeting w/extension staff • Guided by principal areas for improvement identified in LQAS surveys: provide recommendations to guide the team in second half of the project and inform planning

19. PDA Experience/Lessons learnt (3) • PDA to be easy to carry in the pocket of the riding gear • PDA contained automatic skip patterns • Of completeness of the data, all required questions were responded to • a reminder in most required questions in the PDA which could beep • Once an electronic questionnaire was filled in, data is instantly stored and possibilities were built-in to review the data • data is clear in terms of spellings and uniformity • Data entry time is obviously reduced to zero

20. LQAS & the MTE Thank You !