1 / 19

Conditional cash transfers: Evaluation designs and impacts

Conditional cash transfers: Evaluation designs and impacts. Michelle Adato and John Hoddinott International Food Policy Research Institute Evaluations of Social Cash Transfer Schemes in Africa Washington, DC September 14-15, 2009. Introduction. What are CCTs? What are their goals?

HarrisCezar
Download Presentation

Conditional cash transfers: Evaluation designs and impacts

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Conditional cash transfers: Evaluation designs and impacts Michelle Adato and John Hoddinott International Food Policy Research Institute Evaluations of Social Cash Transfer Schemes in AfricaWashington, DCSeptember 14-15, 2009

  2. Introduction • What are CCTs? • What are their goals? • Where are they? • A critical feature of these is the careful attention paid to their evaluation, using quantitative and qualitative methods

  3. Quantitative Evaluations In the context of CCTs in Latin America, quantitative evaluations have performed two principal functions Inform on how well a program works from the perspective of the beneficiaries and others • Does it go to those who are its intended recipients (targeting) • Do people understand how program works? What is required of them - their rights and responsibilities • Do people receive payments in a timely fashion? Do they get all their payments? • Are there constraints that prevent some recipients from receiving what they are entitled to?

  4. Quantitative Evaluations, cont’d Can changes in outcomes – such as the wellbeing of recipients -be CAUSALLY linked to the program • The critical word here is “CAUSAL” • Establishing causality requires accounting for: • The counterfactual – what would have happened in the absence of the program • Selection bias – the assessment of impact is not biased by the mechanism by which some individuals/households were chosen to receive benefits

  5. Double Difference Method: Diagrammatic Household Income Follow-up Beneficiaries ΔT Baseline Controls Year 0 Year 2 TIME

  6. Double – Difference: A second look

  7. Quantitative Evaluations:Addressing Selection Bias • Evaluations of CCTs in Latin America and elsewhere have used three approaches to addressing selection bias: • Randomization: In the evaluation of PROGRESA, as part of the process of phasing in the program, localities were randomly selected to receive program benefits starting in 1998, while others were delayed until 2000. • Regression Discontinuity Design: This method has also been used to evaluate a CCT in eastern Turkey. It is used when there is a “well-defined cut-off” such as a means test or proxy means test that separates those who receive benefits from those who do not. Those beneficiaries “just above” the • Matching methods: This method has been used to evaluation a CCT called BolsaAlimentação as well as the Productive Safety Nets Programme in Ethiopia. Beneficiary households are “matched” to non-beneficiaries with similar observable characteristics. It works best when the selection rules allow for discretion (for example, when community based targeting is used) and when access to the program is rationed

  8. Quantitative Evaluations: Summary • When correctly implemented, quantitative evaluations: • Inform program implementers about what works and what does not work • Inform implementers, policy makers and funders about whether the program has led to measureable improvements in wellbeing • More generally, rigorous and careful evaluations provide a transparent means of assessing program effectiveness. 186 control localities;

  9. Two IFPRI evaluations • PROGRESA in Mexico: • 320 treatment localities; 186 control localities • 24,077 households • Compared > 650 variables, locality and household • Surveys 2x/year for 3 years • Focus groups with 160 Bens/Non-bens and promotoras from 70 localities • Red de Protección in Nicaragua: • 21 treatment localities; 21 control localities • 1,581 households • Qualitative: Ethnographic methods in 6 localities with 120 households

  10. What do Surveys measure in CCT evaluations? • Impacts, e.g. • Participation in health services • Illnesses • Level and composition of expenditure • Diet quality and diversity • Nutritional status • School enrollment, attendance, continuation, achievement • Child and adult labor • Intra-household decision-making • Costs • Supply quality

  11. Types of questions answered through qualitative research • How do CCTs affect gender relations? • Are women getting pregnant to get benefits? • How do NGO methods affect learning and program performance? • Have attitudes toward education and health changed? • Are teachers marking children present when they are absent or lowering grade-passing standards? • What are the implementation bottlenecks? • Is the program affected by politics? • Why do people choose not to participation in services, despite the cash incentive?

  12. Impacts of Conditional Cash Transfers on Education (Enrollment) (Schultz 2001; Skoufias 2005; IFPRI 2003; Maluccio and Flores 2005; Filmer and Schady 2006; Ahmed 2006; Khandker, Pitt, and Fuwa 2003; Ahmed et al. 2007)

  13. Turkey qualitative study: Reasons why children do not attend school, by region (Adato et al. 2007)

  14. Comparing demand and supply-side interventions • Primary school enrollment in Bangladesh (ROSC) • Demand + supply: +8.9% pts • Supply only: no impact • Children under 3 in Honduras (PRAF) who visited health-care provision units: • Demand-side only: + 21 percentage points • Demand + supply: + 15 percentage points • Supply only: no impact (poor implementation) (IFPRI 2003; Ahmed 2006)

  15. Supply side impacts • Mexico: Higher impacts in schools with • lower student-teacher ratios • Larger annual budgets • More telesecundarias in area • Nicaragua: Higher impacts where: • More autonomous (flexible) schools • Areas with poorer initial supply (supply side variables) Behrman, Parker and Todd (2009); Behrman et al. (2006); Maluccio, Murphy and Regalia (2006)

  16. Sustainability • Mexico: • The longer the exposure, the higher the impact • +12 cents/peso invested in microenterprise and agricultural activities=17.5% rate of return. • Nicaragua: • Large impact on children in transition to 6th grade • High health service utilization 8-10 months after program ended • Little effect on investment (apart from HC) (Behrman, Parker and Todd 2009; Gutierrez et al. 2005 Cruz et al. 2005; Gertler, Martinez and Rubio-Codina 2006; Maluccio and Flores 2005; IFPRI 2004)

  17. Average impact on % of children age 7-13 who attend regularly 1-4 grades Phase I (2000-02) 18 p.p.* Impact: Phase II (2002-04) 10 p.p.* Red de Protección Social, Nicaragua (Maluccio and Flores 2005)

  18. Impact on % of children under 5 who had attended preventive growth monitoring in the previous six months Phase I (2000-02) 16 p.p.* Impact: Phase II (2002-04) 15 p.p. RPS, Nicaragua (Maluccio and Flores 2008)

  19. Important issues for future evaluations • Long-term impacts • Hard vs. soft vs. no conditions • Disentangling sources of impact • New indicators and more ‘final’ outcomes • Optimum size of transfer to achieve different objectives • Graduation/transitions • Costs and benefits of different targeting strategies • Flexibility in crises • Long-term impacts • Institutional roles and coordination • New indicators: e.g. HIV risk, social welfare

More Related