Assessing the Distributional Impact of Social Programs

1. The World Bank Assessing the Distributional Impact of Social Programs Public Expenditure Analysis and Manage Core Course Presented by: Dominique van de Walle DECRG March 21-24, 2005 Presented to:

2. The evaluation problem • Who gains from the programs? Who uses public services? At what cost? Who benefits from subsidies? Who are the target groups? How should transfers be allocated? 2. How much do they gain? Is there more poverty with or without a policy? How much impact will programs have on poverty?

3. The evaluation problem • Impact is the difference between the outcome indicator with the program and that without it. • However, we can never simultaneously observe someone in two different states of nature • So, while a post-intervention indicator is observed, its value in the absence of the program — the counterfactual — is not. • The essential problem in evaluation is one of missing data on the counterfactual of what would have happened in the absence of the intervention

4. To measure ‘impacts’ rigorously we need ex-post impact evaluation techniques • Econometric: generally need baseline or panel data • Experimental: require randomized assignment • Often we must instead turn to other “quick & dirty” approaches that examine spending "incidence • The most commonly used is Benefit Incidence Analysis (BIA)

5. What is benefit incidence analysis? Step 1: rank individuals by welfare indicator Access to services Step 2: identify usage/participation Utilization Step 3: attribute "gain" or benefit identified by unit cost of providing service Incidence of spending

6. STEP 1: Access to infrastructure in rural Vietnam (% rural population with the infrastructure)

7. Step 2: Participation in public works and a means-tested credit subsidy in Maharashtra, India Consumption expenditure per person

8. Step 3: A typical example of a benefit incidence analysis:

9. Health spending in Kenya, 1992 100 80 Primary Hospital 60 All health Cumulative subsidy/income Income 40 20 0 0 20 40 60 80 100 Cumulative population

10. Advantages of traditional benefit incidence analysis… Easy to do and to present (with caveats) Disadvantages and limitations… Strong assumptions Do not explain incidence outcomes No specific policy implications

11. Traditional benefit incidence analysis may... 1. Wrongly assume that the cost of provision reflects the benefit to user 2. Be sensitive to method of ranking households in the original position spatial prices, comprehensiveness of welfare indicator, demographics

12. The welfare measure matters for primary education in Ghana 100 80 Per capita expenditure 60 Adult equivalent Cumulative subsidy expenditure 40 20 0 0 20 40 60 80 100 Cumulative population

13. How quintiles are defined matters for health in Ghana 100 80 60 Household quintiles Population quintiles Cumulative subsidy 40 20 0 0 20 40 60 80 100 Cumulative population

14. Traditional benefit incidence analysis may... 3. Mispecify the counterfactual: BIA ignores behavioral responses ex: figure of transfers in Yemen • Conclusions about targeting & incidence depend on how the counterfactual is defined

15. Distribution of public and private transfers in Yemen, 1998, by deciles of per capita expenditures excluding transfers (annual YR per capita)

16. Distribution of public and private transfers in Yemen, 1998, by deciles of per capita expenditures including transfers (annual YR per capita)

17. Traditional benefit incidence analysis may... 4. Give an incomplete picture of welfare effects how did other dimensions of welfare (eg health literacy, nutrition) improve as a result of subsidies? 5. Be unable to assess some important public goods and services eg safe water, sanitation, vector control, physical infrastructure

18. Traditional benefit incidence analysis may... • Ignore general equilibrium & indirect effects on poor eg indirect benefits from tertiary education 7. Confound average and marginal incidence Distribution of gains

19. 7. Average versus marginal incidence • Standard benefit incidence estimates the distribution of average incidence at one point in time • This can be deceptive about how changes in public spending will be distributed • Marginal incidence is an example of a behavioral incidence analysis where one measures the incidence of actual increases or cuts in program spending

20. How will gains from social program expansion be distributed across groups? • Non-poor often capture benefits of (even targeted) social programs information, incentive & political problems make perfect targeting hard • But, program capture by non-poor can differ according to how costs & benefits of participation vary with program scale eg fees, opportunity costs of time, transport costs etc

21. How will gains from social program expansion be distributed across groups? Model 1: Early capture by non-poor: net gains to non-poor are positive initially, but fall with program expansion Model 2: Late capture by non-poor: cost initially too high for the non-poor, but net gains rise over time Average participation rates may be deceptive for inferring how gains & losses from program expansion or contraction will be distributed.

22. One way to identify marginal incidence is to compare incidence across geographic areas with different program sizes Marginal odds of participation (MOP) = increment to group-specific participation rate with a change in overall participation The income group specific MOP is estimated by regressing income group specific participation rate across regions on the average rate for the region. MOP shows incidence of a change in spending

23. Average and Marginal Odds of Primary School enrollment, India 1993-94 Note: odds of enrollment = ratio of quintile-specific enrollment rate to the mean rate.

24. In conclusion • Extreme care is needed when interpreting average incidence and traditional benefit incidence analysis • Beware of reform recommendations based solely on BIA and concentration curves as conventionally calculated.  

25. Q & A