Targeting Programs for Vulnerable Children in Ethiopia

1. Targeting Programs for Vulnerable Children in Ethiopia July 28, 2011 Thomas Pullum Demographic and Health Surveys tpullum@icfi.com

2. My own background When I started this work I was on leave of absence from the Department of Sociology of the University of Texas at Austin, in a temporary two-year position with the U.S. Agency for International Development (USAID) in Washington, D.C. In January 2011 I retired from UT (after 40 years as an academic; am now a professor emeritus) and the move to the D.C. area became permanent. On May 31 I became director of research at the Demographic and Health Surveys, a USAID-funded project based in Calverton, Maryland. The contractor is ICF Macro, a division of ICF International.

3. This presentation is based on a draft paper Co-authors (although thus far I have asked them for little help other than their buy-in), in alphabetical order • Kiersten Johnson, DHS, because of her work on the DHS Wealth Index • Kenneth Land, Duke University, because of his work on the • Child and Youth Well-Being Index • Beverly Nyberg, U.S. Department of State / Office of the Global AIDS Coordinator, because of her work on the Child Status Index

4. Two parts to this presentation Part 1: Origins Part 2: Strategy and results

5. Part 1: Origins This part is non-technical but I hope it will be helpful to have some history and context.

6. Origins of this work From December 2009 to the end of May, 2011, I worked at USAID, Bureau for Global Health, as a Fellow in the Global Health Fellows Program (GHFP), a staffing mechanism that is currently contracted to the Public Health Institute, in Oakland, CA, with some participation by the Tulane and Harvard Schools of Public Health. www.ghfp.net

7. Origins of this work • I was “senior technical advisor for monitoring and evaluation of programs for vulnerable children” within a five-person “secretariat” located within the office of the Assistant Administrator of the Bureau for Global Health. • The purpose of the secretariat was to implement Public Law 109-95, and I was to do M&E related to that. • Public Law 109-95 was passed by Congress in 2005 with the requirement that U.S. Government programs for vulnerable children in developing countries be better coordinated.

8. PL 109-95 Activities and Resources An annually updated database of about 1900 ongoing projects in 109 countries, total of about $2.6b Database of national-level indicators from the U.S. Government, UN agencies (UNAIDS, UNICEF, WHO), and NGOs Hundreds of online links, references, resources on “best practices” Inter-agency working group, newsletter, coordination efforts See website at www.hvcassistance.org

9. Construct a “vulnerability index” • I did not invent this name. Constructing such an index was part of my Scope of Work for the two-year assignment. It is based on the idea that children who are vulnerable tend to be vulnerable in many ways, not just one. • Earlier work related to vulnerability that I built upon: • DHS Wealth Index • Child Status Index • Child and Youth Well-Being Index

10. DHS Wealth Index Developed by Shea Rutstein and Kiersten Johnson at DHS, with World Bank consultation • Takes a large number of household-level variables, e.g. on household possessions, source of water, sanitation, building materials, and synthesizes them with principal components analysis (PCA) into a single composite index. The underlying variables and the weights given to them vary from one country to another.

11. Child Status Index Developed mainly by Measure Evaluation and OGAC (Office of the Global AIDS Coordinator). • Within a community selected for interventions, trained field workers visit all households, rating all children in terms of four domains, two indicators per domain, on a scale of 1 to 4 for each indicator. The average of all indicators is the basis for selecting specific households and children for the interventions, and subsequently measuring impact.

12. Child and Youth Well-Being Index Developed by Ken Land and colleagues at Duke. Well-being is the opposite of vulnerability. • Countries are the units of analysis. All indicators are national-level. Used mainly to track well-being in developed countries over time. • Several conceptual domains, with several indicators in each domain. • Principal components analysis is used to establish that the indicators and domains align and are positively associated. However, the PCA weights are not used further. The indicators are averaged within domains and then the domain means are averaged to get the final value.

13. Part 2: Strategy and results

14. Constructing a Child Vulnerability Index and Applying it to Ethiopia • What are the criteria for a vulnerability index? • What do “vulnerable” and “vulnerability” mean? • Main conceptual and statistical problems • Source of data • Indicators • Domains • Levels of aggregation • Statistical validation • Use for targeting • Calculation of a denominator for gauging coverage

15. Criteria for a Child Vulnerability Index • Must be based on a coherent conception of vulnerability • Must combine multiple types or domains of vulnerability • Must be possible to validate statistically, i.e. to show scalability with real data • Must be based on data that are readily available in developing countries • Must be useful for describing the vulnerability of EITHER individual children OR aggregates of children (for me, the latter)

16. What does “vulnerable” mean? Merriam-Webster: 1: capable of being physically or emotionally wounded 2: open to attack or damage Late Latin vulnerabilis, from Latin vulnerare to wound, from vulner-, vulnus wound; probably akin to Latin vellere to pluck, Greek oulē wound. First Known Use: 1605

17. What does “vulnerable” mean? • Equivalent to “wounded”: in an intermediate condition, partially weakened and at higher risk of future weakness, up to and including premature death. • First entered relevant usage in the late 1990s, in the phrase “orphans and vulnerable children”, or “orphans and other vulnerable children”, abbreviated OVC, and with an explicit connection to HIV/AIDS. • The OVC label is based on an assumption that in a setting with high HIV prevalence, children have higher risk of poor outcomes. The link to HIV has become less crucial in the last couple of years.

18. Major conceptual problem • Problem: Does “vulnerability” refer to risk factors or to bad outcomes? For example, is stunting (short for age) a risk factor or is it a bad outcome? This division into risk factors and bad outcomes is the root cause of the regression hangup. • Solution: Don’t distinguish between risk factors and bad outcomes. Pool them. Use all available information about child protection and welfare, from the household or the child, regardless of its position in a causal model.

19. Major statistical problem • Problem: Most of the information is age-specific. For example, stunting is only given in surveys for children age <5 or girls age 15-17. School attendance is only known for school-age children. How can such information be synthesized? • Solution: Don’t try to calculate a composite measure of vulnerability for individual children. Pool children into groups or geographical areas and calculate the index for those groups or areas.

20. Data • Source of data: Demographic and Health Surveys (DHS, USAID) OR Multiple Indicator Cluster Surveys (MICS, UNICEF) • Components of both survey programs Household survey with listing of all persons Survey of women 15-49 Often a survey of men, sometimes starting at age 15 Additional information about children born in the past five years Sometimes information about HIV in the household • Unfortunate deficiency: Because these are household surveys, they omit children living outside of households—who are among the most vulnerable children

21. Possible indicators of vulnerability • Several possible indicators in these surveys but not really a huge number. • To each child age 0-17 in the household survey, we attach whatever information is available; some of it is age-specific. • Code each individual-level variable “1” if the child is in the “bad” category, “0” otherwise, except “missing” if the item is not applicable or no response, in which case the child contributes no information on that indicator. • The indicators listed here are subject to change or even subject to a country-specific selection process.

22. Domains of potential vulnerability Seven domains, largely consistent with those included in other related indices: • Affected by HIV/AIDS • Parental death • Material deprivation • Health • Education • Nutrition • Child protection As much as possible, the indicators are separate from interventions that are already in place

23. Indicators within domains • Affected by HIV/AIDS Anyone in the household is HIV positive • Parental death Mother has died Father has died • Material deprivation Low value for the household on the continuous wealth index • Health Child death in household in past five years Recent illness

24. Indicators within domains (continued) • Education Household head has no education Child age 6-11 is not in school Child age 12-17 is not in school • Nutrition Child is stunted (short for age) Child is wasted (low weight for height) • Child protection No birth registration Early fertility Working without pay

25. Units of analysis—Important! • The indicators refer to children, households, or household heads, and are attached, as applicable, to children age 0-17 in the household file. • However, a value of the overall index is NOT calculated for individual children or for households. • It is only calculated for spatial aggregates such as clusters or regions • Results for low level aggregations such as clusters would never be used, except as part of the validation of the index.

26. Combining or synthesizing the indicators / domainswithin a geographic area 1. For a given indicator, calculate the proportion of children 0-17 who are in the “bad” category of the indicator, limited to the children for whom the indicator can be calculated. 2. Combine indicators and then domains by averaging. The average for the domains will be the index value. Example: in an area, there are 1000 children age 0-4 and girls 15-17 for whom valid measurements of height and weight were taken. 400 of them are stunted. The score for stunting will then be 400/1000 = 0.40 In the same area, 300 of the children are wasted. The score for wasting will then be 300/1000 = 0.30. These are the two indicators for the nutrition domain. The domain score will be (0.40 + 0.30)/2 = 0.35.

27. Validate or justify the combining of domains 1. Calculate the correlations among the seven domain scores (also use principal component analysis and examine the first principal component). If the domain scores have strong positive correlations, then combine them by just averaging them, within each of the geographical areas. 2. Strong positive correlations imply that the domains of vulnerability overlap, and interventions in one domain can efficiently be combined with interventions in another domain, by the averaging step. Example: Nutrition and Education. Say that both nutrition and education tend to be poor in some areas and good in other areas. It will then be more efficient to develop programs that treat both problems in the same location. The average of the Nutrition and Education domains would be calculated for different regions to find the regions with the highest combined scores.

28. Grouping of domains in Ethiopia In Ethiopia, the seven domains fall into two groups, within which the domains are strongly and positively associated: • First group: Affected by HIV/AIDS and Parental Death. Approximately the OVC criteria. Summarize with the arithmetic average and call it the OVC Index. • Second group: Material deprivation, Health, Education, Nutrition, Child protection. (Health has the weakest correspondence with the others.) Summarize with the arithmetic average and call it the Child Vulnerability Index. • In Ethiopia, these two groups and indices are NEGATIVELY associated. It is not expected that this pattern will be observed in all countries.

29. Interpreting the index Interpreting the value of the index (either index) for a specific geographical area such as a region or district: • It is the average proportion of children, for the indicators and domains specified in that index, and for the children to whom the indicators apply, who have the specified disadvantages. • Note: because so many of the indicators are age-specific, we can only talk about associations of the indicators within groups or areas. We cannot measure association among vulnerabilities at the level of the individual child. • Note: The various proportions are averaged rather than added. If we added them, the index would go up if we just added more indicators.

30. Intensity and magnitude of vulnerability • The index (from either group of domains) can be converted to an estimated number of children. Multiply the index (thinking of it as a proportion) by the number of children age 0-17 in the region, district, etc., to get the number of children who have the specified disadvantages. • The index describes the intensity of vulnerability / deprivation. • In Ethiopia as a whole, the OVC Index is 0.046 (or 4.6%) and the Child Vulnerability Index is 0.370 (37.0%). • The estimated number of children describes the magnitude of vulnerability / deprivation. In Ethiopia as a whole, the OVC magnitude is 1.6 million children and the Child Vulnerability magnitude is 15.3 million (out of a total of 40 million children).

31. Using the index to prioritize the location of interventions The regions and districts of a country can be ranked in four ways: • Intensity of OVC Index • Magnitude of OVC Index • Intensity of Child Vulnerability Index • Magnitude of Child Vulnerability Index As a guide to selecting locations, specifying targets, and combining interventions to improve effectiveness. Two of these tables are given next.

32. Levels and rankings of the Child Vulnerability Index in the urban and rural portions of the regions of Ethiopia. 2005 DHS survey of Ethiopia. Urban areas Rural areas region CV_Index rank CV_Index rank Tigray 0.1974 16 0.4164 3 Afar 0.2009 15 0.5111 2 Amhara 0.1867 18 0.3977 5 Oromiya 0.2014 14 0.3815 7 Somali 0.3142 12 0.5226 1 Ben-Gumz 0.1729 19 0.3736 8 SNNP 0.1900 17 0.3571 9 Gambela 0.2130 13 0.3878 6 Harari 0.1518 22 0.3513 10 Addis Ababa 0.1568 21 0.3156 11 Dire Dawa 0.1572 20 0.4068 4

33. Estimated numbers of children age 0-17 in need of assistance, using the Child Vulnerability Index, in the urban and rural portions of the regions of Ethiopia. 2005 DHS survey of Ethiopia. The total number is 15,319,247. Urban areas Rural areas region CV_Index rank CV_Index rank Tigray 68,899 12 908,920 4 Afar 9,188 18 201,238 7 Amhara 116,470 10 3,554,727 2 Oromiya 222,784 6 5,309,300 1 Somali 63,231 13 818,689 5 Ben-Gumz 3,868 21 124,037 9 SNNP 86,005 11 3,099,450 3 Gambela 2,643 22 37,517 14 Harari 5,763 20 15,073 16 Addis Ababa 130,949 8 6,470 19 Dire Dawa 11,892 17 27,058 15

34. Next steps The draft paper has been reviewed by the PL 109-95 team; has been reviewed by Demographic and Health Surveys staff; has been reviewed by the OVC and Child Protection staff at UNICEF; will be revised and extended to other countries. Your comments will be most welcome, now or later by email (tpullum@icfi.com) Thank you!