Assessing the Impact of the Ishraq Program in Egypt

1. Assessing the Impact of the Ishraq Program in Egypt Ghada Barsoum, PhD Senior Program Manager Poverty, Gender and Youth Program Population Council International Initiative for Impact Evaluation Conference April 2009

2. Ishraq: Basic Facts • Targets out-of-school girls (12-15) • In rural Upper Egypt: Lowest human development indicators: 26% of girls receive no schooling (ELMPS06); unpaid family enterprise workers; early marriage and childbearing; FGM • Intervention lasts for 20 months • Girls meet 4 times weekly for 3 hours

3. Ishraq: Basic Facts • First piloted in 2001 by four Int’l NGOs: Save the Children; CEDPA (the Center for Development and Population Activities); Caritas; Population Council • Government partner with an increasing role: National Council of Youth (NCY) to allocate specific hours in youth centers to be used as “safe spaces” for girls • Local NGOs implementing the model and mobilizing communities on the village level

4. Ishraq: Basic Facts Program Components: • Literacy (Learn to be Free, Caritas): Girls sit for the literacy exam and are given the opportunity to be mainstreamed into the formal educational system (grade 7) – Program provides help in issuing birth certificates (required for school enrollment) • Life-Skills (RH, Basic: livelihood, nutrition, girls rights .. (CEDPA) • Sports and Recreation (PC) • Financial Literacy (PC) (new) • Free Meals and food rations (new)

5. Intervention so far .. • Piloted in 2001 in 4 villages in Minia (N=275); expanded in 2004 to 6 villages in Minia (2nd rounds in 2 villages from the pilot stage) (N=336); expanded in 2006 to 5 villages in Beni Suef (N=289) • Currently in its scale-up and –out phase in 50 villages (2500 girls) and in full partnership with NCY with an institutionalization plan for sustainability

6. Rigorous Impact Evaluation has been Central to Ishraqin both its Pilot and Scale-up Phases

7. Impact Evaluation in the Earlier Phase • Baseline and endline surveys in control and intervention villages • Most of the analysis was descriptive due to limited sample size. Positive outcomes in: • Functional literacy, cognitive skills and continuing schooling; • 92% of those who sat for the literacy exam passed; over half achieved “Excellent” score; 66% (re-)entered the formal education system (compared to a national average of 6% for those who pass literacy exams) • Positive gender norm attitudes; knowledge about nutrition, hygiene, girl’s rights & RH; against FGM

8. Impact Evaluation Plan for the Scale-up PhasePlan and Challenges

9. Combined Methodology • Baseline and endline surveys in control and intervention villages • Combining Monitoring data (such as participation and attendance levels) with impact evaluation data • Qualitative research

10. Self-Selection Bias Concerns • Girls who self-select into the program are more likely to be better off. Selection bias problem results in mixing-up the treatment effect with pre-program difference in attributes. • To estimate the impact of the program we need to know the difference between: (1) the outcomes for participants if they participated in the program and (2) the outcomes had they not participated

11. Pre-analysis Activities Addressing Selection Bias • Girls’ self-selection into participation and recruitment activities • Village selection • Questionnaire design • Data collection plan

12. 1. Recruitment Activities and Selection Bias • Recruitment: debate on whether first-come-first-served or randomization based on hh listing • Rigorous community mobilization to eliminate the impact of: • Geographical proximity • Friends and relatives 2. Improved Ishraq recruitment strategy through visual advertisements; trained promoters; village committees; village-level workshops

13. 2. Village Selection Process • Randomization in the selection of intervention villages from a list of eligible villages. Eligibility criteria: size of out-of-school cohort, presence of a youth center and a school • Statistical power to determine the number of control villages • Selection of control villages from the very same list of eligible villages

14. 3. Questionnaire Design Adding instrumental variables to household and individual characteristics and attitudes • Instruments that affects participation but not directly affect outcomes. • Distance between home and youth center (proxy for costs of participation). + Distance to the village center as an explanatory variable in both the participation and outcome equations to capture access to other services and the effect of the centrality of the household location.

15. 4. Data Collection Plan at Baseline and Endline • A saturation sample of out-of-school girls in intervention and control villages based on listing and mapping of all hhs in village • Design allows us to compare the characteristics of control village and intervention village members; and the characteristics of participants and non-participants in intervention villages • Data from M&E tools to be incorporated into endline dataset

16. Examined Outcomes • Literacy • level of success in passing government-sponsored literacy tests and rates at which girls are mainstreamed back into preparatory (middle) schools. • Attitudes about marriage and childbearing: • Ideal age at marriage • Views about decision-making regarding marriage partners and timing of marriage. • Ideal family size and fertility intentions.

17. Examined Outcomes- CONTD • Knowledge about nutrition, hygiene, and reproductive health. • Attitudes about harmful traditional practices (e.g., FGM) • Social isolation, peer networks, and participation in group or community activities • Gender norms index

18. Qualitative Data Collection • Confirming and enriching findings • Merging the two methodologies in the analysis and write up of results • Open-ended semi-structured interviews with girls joining the project, drop-out cases and their families • Focus group discussions • Informs program design and constitutes an ongoing process

19. Thank You