Benefit Incidence Analysis: the method, limitations and extensions

1. Benefit Incidence Analysis: the method, limitations and extensions Dominique van de Walle World Bank May 2009

2. Questions typically asked by policy makers: Who gains from public programs and policies & how much do they gain? Who uses public services? At what cost? Who benefits from subsidies? Who are the target groups? How should transfers be allocated? Is there more poverty with or without a policy? How much impact will programs have on poverty?

3. 1. Incidence analysis • Benefit incidence analysis (BIA) 2. Introducing behavioral responses

4. To implement basic benefit incidence analysis: Essential data 1. household level data with household or individual level information on: • welfare indicator (income or consumption per person) • utilization or access data 2. unit cost of provision • total (recurrent) spending on the policy or program • minus fees • divided by total beneficiaries

5. Steps in benefit incidence analysis Step 1: Rank individuals by the pre-intervention welfare indicator Income or consumption per person is common ex: Here you can also look at how access to services varies

6. Step 1: Access to infrastructure in rural Vietnam 1993 (% rural population with the infrastructure)

7. Steps in benefit incidence analysis Step 1: Rank individuals by pre-intervention welfare indicator Income or consumption per person is common (Here you can also look at how access to services varies) Step 2: Identify usage/participation Utilization is typically measured from a survey

8. Step 2: Participation in public works and a means-tested credit subsidy in Maharashtra, India Consumption expenditure per person

9. Steps in benefit incidence analysis Step 1: Rank individuals by pre-intervention welfare indicator Income or consumption per person is common Here you can also look at how access to services varies Step 2: Identify usage/participation Utilization is typically measured from a survey Step 3: Attribute "gain" or benefit identified by unit cost of providing service Look at incidence of spending across welfare groups

10. Step 3: Typical examples of benefit incidence analyses:

11. Subsidy per capita by decile, Indonesia 1989 Education Health Rupiahs/year Rupiahs/year

12. Health spending in Kenya, 1992 100 80 Primary Hospital 60 All health Cumulative subsidy/income Income 40 20 0 0 20 40 60 80 100 Cumulative population

13. South Africa -Distribution of VAT exemption Source: Alderman and del Ninno (1997)

14. Advantages of traditional benefit incidence analysis… • Easy to do and to present (with caveats) Disadvantages and limitations… • Strong assumptions • Do not explain incidence outcomes: demand side issues?? supply side issues?? • No specific policy implications Further reading: van de Walle, Dominique, 1998. "Assessing the Welfare Impacts of Public Spending," World Development, 26(3): 365-379

15. Traditional benefit incidence analysis may... 1. Wrongly assume that the cost of provision reflects the benefit to user 2. Be sensitive to method of ranking individuals in the original position spatial prices, comprehensiveness of welfare indicator, demographics

16. The welfare measure matters for primary education in Ghana 100 80 Adult equivalent expenditure 60 Per capita expenditure Cumulative subsidy 40 20 0 0 20 40 60 80 100 Cumulative population

17. How quintiles are defined matters for health in Ghana 100 80 60 Household quintiles Population quintiles Cumulative subsidy 40 20 0 0 20 40 60 80 100 Cumulative population

18. Traditional benefit incidence analysis may... 3. Give an incomplete picture of welfare effects how did other dimensions of welfare (eg health literacy, nutrition) improve as a result of subsidies? 4. Be unable to assess some important public goods and services eg safe water, sanitation, vector control, physical infrastructure

19. Traditional benefit incidence analysis may... 5. Ignore general equilibrium & indirect effects on poor eg indirect benefits from tertiary education 6. Confound average and marginal incidence Distribution of gains

20. Traditional benefit incidence analysis may... 7. Mispecify the counterfactual: BIA ignores behavioral responses ex: transfers in Yemen • Conclusions about targeting & incidence depend on how the counterfactual is defined

21. Distribution of public and private transfers in Yemen, 1998, by deciles of per capita expenditures excluding transfers (annual YR per capita)

22. Distribution of public and private transfers in Yemen, 1998, by deciles of per capita expenditures including transfers (annual YR per capita)

23. Introducing behavioral responses into incidence analysis How participants and others respond to programs can matter to distributional outcomes Yet, traditional benefit incidence analysis ignores behavioral responses • Some simple tools can help incorporate behavioral responses into the analysis of the incidence of public spending or policy changes. Further reading: van de Walle, Dominique, 2003. "Behavioral Incidence Analysis of Public Spending and Social Programs." In Luiz A. Pereira da Silva and François Bourguignon, eds., The Impact of Economic Policies on Poverty and Income Distribution: Evaluation Techniques and Tools. World Bank and Oxford University Press.

24. Two types of responses: • By the participants and those they interact with • By administrative or political agents • Key issue for all incidence analysis is how to define the counterfactual: what the welfare of beneficiaries would be without the program. • ·

25. Assumptions about behavior matter We need to assess incidence relative to the counterfactual — without public spending. An appropriate indicator is needed to identify the poor. Conventional benefit incidence analysis usually assumes the without‑intervention position to be the welfare indicator less the value of the benefits Yet, programs affect savings, labor effort, schooling choices and private transfers received

26. Assessing behavioral responses to transfers • What would welfare have been without government intervention? • Ideally, one would subtract transfers but add in the replacement income households would have had without the intervention

27. Most direct approach is to see how much consumption changes when benefits are received • Estimate the marginal propensity to consume • out of social income (PCSI); • Determine net gain to consumption from social transfers; and • Construct the counterfactual consumption level without intervention

28. Consumption model for Vietnam using panel data Consumption of h’hold i at time t (=1993, 1998): for public transfers (Tit), observed household characteristics (Xit), time varying (t) and time invariant (i) factors.

29. Marginal incidence analysis • Standard benefit incidence estimates the distribution of average incidence at one point in time • This can be deceptive about how changes in public spending will be distributed. • Marginal incidence is an example of a behavioral incidence analysis where one measures the incidence of actual increases or cuts in program spending

30. 1. Marginal incidence analysis usingsingle cross-sectional data • How will gains from social program expansion be distributed across groups? • Non-poor often capture benefits of (even targeted) social programs • information, incentive & political problems make perfect targeting hard • But, program capture by non-poor can differ according to how costs & benefits of participation vary with program scale • eg fees, opportunity costs of time, transport cost etc. Further reading: Lanjouw, Peter and Martin Ravallion, 1999. “Benefit Incidence and the Timing of Program Capture,”World Bank Economic Review 13(2): 257-74

31. Model 1:Early capture by non-poor: net gains to non-poor are positive initially, but fall with program expansion Model 2:Late capture by non-poor: cost initially too high for non-poor, but net gains rise over time Average participation rates may be deceptive for inferring how gains & losses from program expansion or contraction will be distributed

32. One way to identify marginal incidence is to compare incidence across geographic areas with different program sizes Average odds of participation = ratio of group specific average participation rate to overall average Marginal odds of participation (MOP) = increment to group-specific participation rate with a change in overall participation MOP shows incidence of a change in spending

33. How can it be estimated?  Regress income group specific participation rate across regions on average rate for region to get income group specific MOP.

34. Average and Marginal Odds of Primary School enrollment, India 1993-94 Note: odds of enrollment = ratio of quintile-specific enrollment rate to the mean rate.

35. Method requires: • Cross-sectional household survey • With data on household level program participation and welfare • Sufficient regional dissaggregation and variance in participation. Main concern: • If there are important state-level differences in the propensity to reach the poor that are correlated with levels of social spending

36. 2. Marginal incidence analysis using repeated cross-sectional data Assessing incidence at two or more dates may be more robust to heterogeneity in local political factors. Average incidence (Ejt / Et) gives the average share of total participation or spending going to quantile j at date t = 1,2 The change in quantile specific participation or share of spending can then be represented by:

37. Safe Water Coverage, Malaysia (% of households) 1985 1990 1990 1985 (6) (5)

38. The approach can be applied to education, health, social transfers & other public spending programs for which participation at household level can be identified and a benefit value attributed. It requires 2 cross sectional surveys with comparable data The important point is that there may be a big difference between average incidence at a point of time and the marginal incidence Further reading: Hammer, Jeffrey, Ijaz Nabi and James Cercone, 1995, “Distributional Effects of Social Sector Expenditures in Malaysia, 1974 to 1989” in van de Walle and Nead (eds.) Public Spending and the Poor: Theory and Evidence. London: The Johns Hopkins University Press. van de Walle, Dominique, 1994,"The Distribution of Subsidies Through Public Health Services in Indonesia, 1978-87", The World Bank Economic Review, 8: 279-309, May 1994.

39. 3. Marginal incidence analysis using panel data • Panel data allows an exploration of dynamic marginal impacts of spending using benefit incidence with data that follows the same households over time. • Method developed to examine how well the social safety net protects vulnerable households from falling into poverty versus how well it promotes households out of poverty • The static average incidence is uninformative about this and • may be deceptive about how much outlays, coverage, and changes over time are correlated to poverty related shocks and changes in exogenous variables.

40. Testing the safety net in Vietnam 1993-1998 • Poverty fell dramatically • Survey data indicate a 127% proportionate increase in real per capita outlays on transfers between 1993-1998 • Was this expansion pro-poor? Did it perform a safety net function? Were changes in transfers responsive to poverty-related shocks? • Idea: simulate a counterfactual joint distribution of the welfare indicator overtime without changes in transfers Further reading: van de Walle, Dominique, 2004, “Testing Vietnam’s Safety Net,”Journal of Comparative Economics, 32 (4): 661-679

41. Incidence of changes in transfers by initial consumption and changes in consumption over time, Vietnam 1993-98

42. There is little sign that the system responded to consumption shocks • % beneficiary h’holds is relatively uniform across cells • Neither starting out poor, nor experiencing negative consumption shocks, appear to have elicited a response from social programs • 32% of those with highest initial consumption and the highest gains were beneficiaries compared to 34% of the worst off in both respects • If anything, the per capita transfer amounts increases with initial and rising welfare • The smallest amount went to the most needy

43. What role did transfers play in the reduction in poverty? • Use panel structure to assess how well the safety net protected from falling into poverty and promoted out of poverty • H'holds are classified into 4 groups: whether poor or non-poor in both years, and whether escaped or fell into poverty over the period. • 27% of the population escaped poverty; • 5% fell into poverty; • 34% were persistently poor and • 35% never poor.

44. Baseline discrete joint distribution of actual consumption (%) Considerable persistent poverty

45. Transfers had negligible impact on poverty Counterfactual joint distribution without transfers (%)

46. Low spending, low coverage and poor targeting explain the negligible impact of transfers on poverty • Without transfers, 1 & 2 additional % of the population would have been poor in 1993 & 1998 respectively • Changes in transfers enabled 1% to escape poverty, & protected 1% from falling into poverty • Social transfers had little bearing on poverty reduction 1993-98 • Nor did they protect those who faced falling living standards

47. Conclusions • Extreme care is needed when interpreting average incidence and traditional benefit incidence analysis • Beware of reform recommendations based solely on BIA and concentration curves as conventionally calculated. 

48. Conclusions (cont.) Ignoring behavioral responses to public spending can yield deceptive assessments of incidence by: • incorrectly assigning beneficiaries to the pre-intervention distribution. • ignoring the influence of the political economy on the assignment of beneficiaries

49. Conclusions (cont.) Some relatively simple methods exist that can help address the deficiencies of non-behavioral incidence analysis: • with household panel data, one can exploit changes in program spending over time to obtain estimates of the impact of transfers on incomes robust to potential endogeneity of the program assignment across units • Then, one can work out what participant incomes would have been without the program, and so estimate the incidence of spending relative to that counterfactual.