The impact evaluation of the progresa oportunidades program of mexico
This presentation is the property of its rightful owner.
Sponsored Links
1 / 85

The Impact Evaluation of the PROGRESA/Oportunidades program of Mexico PowerPoint PPT Presentation


  • 100 Views
  • Uploaded on
  • Presentation posted in: General

The Impact Evaluation of the PROGRESA/Oportunidades program of Mexico. Emmanuel Skoufias The World Bank PRMPR April 2007. Why are CCT programs attractive?-1. CCT can provide the foundation of a comprehensive Poverty Alleviation and Social Protection Policy

Download Presentation

The Impact Evaluation of the PROGRESA/Oportunidades program of Mexico

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The impact evaluation of the progresa oportunidades program of mexico

The Impact Evaluation of the PROGRESA/Oportunidades program of Mexico

Emmanuel Skoufias

The World Bank

PRMPR

April 2007


Why are cct programs attractive 1

Why are CCT programs attractive?-1

  • CCT can provide the foundation of a comprehensive Poverty Alleviation and Social Protection Policy

    • Induce investments by poor on human capital

    • Can mitigate short-run macroeconomic shocks

    • Can facilitate the phase-out of price subsidies and/or other less effective programs

    • Can serve as a basic social safety net system available to ALL households (complementary to the safety net system accessible through employment in formal sector)


Why are cct programs attractive 2

Why are CCT programs attractive?-2

  • CCT can achieve a significant redistribution of income in favor of the poor under tight fiscal conditions

    • Oportunidades was initiated in the context of a short-run economic crisis (1994-95) and was designed as medium-term response to the crisis

    • Oportunidades gradually replaced generalized food subsidies with direct monetary transfers


Why are cct programs attractive 3

Why are CCT programs attractive? -3

  • CCT exploit the complementarities among health, education, and nutrition.

    • Coordination: Promote coordination of poverty alleviation efforts among Gov’t ministries (education, health, nutrition)

    • Synergy: simultaneous provision of health, education and nutrition benefits to all the beneficiaries.


Why are cct programs attractive 4

Why are CCT programs attractive?-4

  • Co-responsibility: The beneficiaries need to take specific actions or else they do not receive benefit

  • CCT have potential of leading to lasting improvements in the well being of the poor

    • Evidence from rigorous impact evaluation of Oportunidades in Mexico Familias en Accion in Colombias and other countries


Dual objectives of cct

Dual Objectives of CCT

  • Long-run poverty alleviation through investment in human capital (i.e., education, health and nutrition)

    • Early Interventions have much higher returns over life-cycle

  • Short-run effect on poverty through cash transfers


How cct try to achieve their objectives 1

How CCT try to achieve their objectives?-1

  • Targeting (geographic/household-level)

    • Oportunidades combines geographic/village-level with household level targeting within villages

  • Simultaneous intervention in 3 key sectors (synergy)


How cct try to achieve their objectives 2

How CCT try to achieve their objectives?-2

  • Conditioning cash transfers to regular school attendance and visits to health centers

  • Cash transfers given to mothers

  • Parallel support on Supply Side (schools & health centers)


Controversial aspects of oportunidades 1

Controversial aspects of Oportunidades-1

  • Why grant direct monetary transfers instead of food in-kind, vouchers, or improving supply side of services.

    Distribution of large volumes of food free of charge can inhibit the development of private commercial channels and create unfair competition with marginal producers in the area

  • Why target on the extreme/structurally poor and not include all?

  • Setting new selection criteria: Why not than start from beneficiary lists of existing programs or obtaining the roster of beneficiaries from community proposals


Controversial aspects of oportunidades 2

Controversial aspects of Oportunidades-2

  • Creating a single national roster of beneficiaries

  • Giving transfers directly to individuals rather than to communities

  • Having unique, non-discretionary rules for the whole country rather than allowing flexibility for local initiatives and conditions in each state


Controversial aspects of oportunidades 3

Controversial aspects of Oportunidades-3

  • Granting benefits to women, given potential family conflicts

  • Having possible impact on fertility (since benefits are linked to family demographics)

  • Size of cash transfer

  • the definition of family co-responsibilities and their certification (might generate additional workload for teachers and medical personnel)


How the controversial aspects of oportunidades were managed

How the controversial aspects of Oportunidades were managed

  • Piloting

  • Expansion of the program in phases

  • Independent and rigorous evaluation (targeting, impact of the program on health, education, nutrition, social relations, women’s status etc.)

  • Monitoring

  • Operational evaluation of the program

  • Cost analysis


Why evaluation

Why Evaluation?

  • Economic Reasons

    • Improve design and effectiveness of the program

    • Comparing program impacts allows G to reallocate funds from less to more effective programs and thus to an increase in Social Welfare

  • Social Reasons (increases transparency & accountability)

  • Political Reasons

    • Credibility/break with “bad” practices of past


Cct programs like oportunidades expanding

CCT programs (like Oportunidades) Expanding

  • Brazil: Bolsa Familia=Bolsa Escola, Bolsa Alimentacao & Programa de Erradicaçao do Trabalho Infantil (PETI)

  • Colombia: Familias en Acción

  • Honduras: Programa de Asignación Familiar (PRAF)

  • Jamaica: Program of Advancement through Health and Education (PATH)

  • Nicaragua: Red de Protección Social (RPS)

  • Turkey

  • Ecuador: Bono Solidario

  • Argentina:

  • Bangladesh: Food for Education


Oportunidades previously called progresa

OPORTUNIDADES(previously called PROGRESA)

  • Large program covering rural and marginal urban areas

    • In 2004: 5 million families or 25 million individuals

    • In 2004: budget of US$ 2.5 billion or 0.3% of GDP


Program description benefits

Program Description & Benefits

  • Education component

    • A system of educational grants (details below)

    • Monetary support or the acquisition of school materials/supplies

      (The above benefits are tied to enrollment and regular (85%) school attendance)

    • Improved schools and quality of educations (teacher salaries)


Program description benefits1

Program Description & Benefits

  • Health and Nutrition Component

    • basic package of primary heath-care services

    • Food support (cash)

    • nutritional supplements: 6 packs/child/mo; 20% of caloric requirements and 100% of necessary micronutrients)

      (The above benefits are tied to regular visits to health-service centers).

    • Information and training

    • Improved supply and quality of health services (medicine availability etc.)


Program description benefits2

Program Description & Benefits

  • Average benefit received by beneficiary households: or 20% of the value of consumption expenditure before program

  • About half transfer is the cash transfer for food and the rest from the school-related cash transfer


Progresa oportunidades evaluation design

PROGRESA/OPORTUNIDADES: Evaluation Design

  • EXPERIMENTAL DESIGN: Program randomized at the locality level (Pipeline experimental design)

  • IFPRI not present at time of selection of T and C localities

  • Report examined differences between T and C for more than 650 variables at the locality level (comparison of locality means) and at the household level (comparison of household means)

  • Sample of 506 localities

    –186 control (no program)

    – 320 treatment (receive program)

  • 24, 077 Households (hh)

    • 78% beneficiaries

    • Differences between eligible hh and actual beneficiaries receiving benefits

    • Densification (initially 52% of hh classified as eligible)


Targeting

Targeting

  • Step 1: geographical targeting

    • Identify localities of highest marginality

  • Step 2: Household-level targeting

    • Within the localities identified in step 1, conduct a household census and use that census to collect socio-demographic data and information on housing characteristics that is consistent and standard nationwide.

    • Use discriminant analysis on census data to classify household as poor or nonpoor


The impact evaluation of the progresa oportunidades program of mexico

Table: A Decomposition of the Sample of All Households in Treatment and Control Villages


The impact evaluation of the progresa oportunidades program of mexico

Treatment

E (Y)

2DIF

impact

estimate

Control

Before

After


The impact evaluation of the progresa oportunidades program of mexico

Using regressions to get 2DIF estimates:

Limit sample to eligible households in treatment and control and run regression:

  • Y(i,t) denotes the value of the outcome indicator in household (or individual) i in period t,

  • alpha, beta and theta are fixed parameters to be estimated,

  • T(i) is an binary variable taking the value of 1 if the household belongs in a treatment community and 0 otherwise (i.e., for control communities),

  • R2 is a binary variable equal to 1 for the second round of the panel (or the round after the initiation of the program) and equal to 0 for the first round (the round before the initiation of the program),

  • X is a vector of household (and possibly village) characteristics;

  • last term is an error term summarizing the influence random disturbances.


Evaluation tools

Evaluation Tools

Formal surveys

(Semi)-structured observations and interviews

Focus groups with stakeholders (beneficiaries, local leaders, local PROGRESA officials, doctors, nurses, school teachers, promotoras)


Progresa evaluation surveys data

PROGRESA Evaluation Surveys/Data


Progresa evaluation surveys

PROGRESA Evaluation Surveys

Additional Info Sources

School & clinic survey

School and clinic administrative data

Nutrition survey conducted independently by Min. of Health and INSP

Student achievement test scores by Min of Education

Record of payments distributed to beneficiary households


Topics of progresa s evaluation

Topics of PROGRESA’s Evaluation

Targeting accuracy and impact on poverty

School enrollment, attendance, child labor, achievement scores

Health and utilization of health facilities

Child Nutrition

Household Consumption & Nutrition


Topics of progresa s evaluation cont d

Topics of PROGRESA’s Evaluation cont’d

Operation of the program and perceptions of stakeholders

Cost-Analysis and Cost Effectiveness

Status of women, community relations

Adult labor supply, leisure

Impact on short-run poverty

intrahousehold transfers


Conditional cash transfers adult work incentives and poverty

Conditional Cash Transfers, Adult Work Incentives, and Poverty

Emmanuel Skoufias

The World Bank

[email protected]

Vincenzo di Maro

University College London

[email protected]

November 3, 2006

LACEA meetings Mexico City


Introduction 1

Introduction-1

  • Means-tested CCT programs increasingly popular

  • Their success at reducing current poverty depends on whether, and the extent to which, cash transfers reduce adult work incentives.

  • In a CCT a cash transfer can be considered as having offsetting income and substitution effects.

    • Income effect: The cash transfer increases household income which in turn increases both Consumption and Leisure (decreases adult labor supply).

    • Substitution effects: associated with meeting the conditionalities of the program. Direct and indirect time costs that depend on the substitution possibilities in the time of family members used in household production.


Introduction 2

Introduction-2

  • Thus, the response of adult labor supply to CCTs can only be determined empirically.

  • Most welfare programs in the US, have explicit disincentives to work.

  • In most CCT programs the level of benefits received is not affected by the work decisions of hh members or the income level of the hh (once the hh is determined as eligible for the program).

  • However, there are other sources of negative incentives on adult labor supply for both eligible and non-eligible households.

    • Eligible hh may work/earn less so as not to get an income level above the poverty threshold used by the program.

    • Non-eligible hh may work/earn less so as to qualify for the program


Introduction 3

Introduction-3

  • Empirical evidence on how labor supply responds to transfers is scarce. Sahn & Alderman (1996): the labor supply effect of a rice subsidy in Sri Lanka is significantly large.


Objectives 1

Objectives-1

Use data from the PROGRESA 1997-99 evaluation sample to investigate:

  • whether eligibility for program affect adult labor force participation (in the treatment villages)

  • whether the presence of the program affects the labor force participation of adults from non-eligible households (in the treatment villages)

  • The effect of eligibility on adult leisure time

  • The impact of the program on poverty measures based on household income.


The impact evaluation of the progresa oportunidades program of mexico

Participation in Labor Force

where

Leisure (from time allocation survey in June 1999:


The impact evaluation of the progresa oportunidades program of mexico

Table 1: Impact (2DIF estimates) on the probability of working among eligible (E=1) adults


The impact evaluation of the progresa oportunidades program of mexico

Table 2: Impact (2DIF estimates) on the probability of working among NON-Eligible (E=0) adults


Tables 1 2 results 2dif 1

Tables 1 & 2 results 2DIF (1):

  • PROGRESA has no significant impact on the labor force participation of adult males and/or females

  • There are no significant/lasting impacts on the allocation of time between salaried and self-employment activities,

  • though the program appears to be associated with a negative, albeit non-significant, effect on self-employment among males


Tables 1 2 results 2dif 2

Tables 1 & 2 results 2DIF (2):

  • The presence of PROGRESA has no significant impact on the labor force participation of adult males and/or females from noneligible households (in the treatment villages)


The impact evaluation of the progresa oportunidades program of mexico

Table 3: Impact (CSDIF estimates) on Leisure of Eligible (E=1) adults


Table 3 csdif 1

Table 3 CSDIF (1):

  • PROGRESA has no significant impact on the leisure time of adult males and/or females


Impact of progresa on poverty 1

Impact of PROGRESA on Poverty (1):

  • Results so far: program has no adverse effects on labor income

  • Effects on total hh income and thus poverty depends on the direct and indirect costs associated with participation in PROGRESA.

  • Participation in PROGRESA

    • (a) income losses form children’s work

    • (b) give up benefits from other programs (DIF, Ninos de Solidaridad, Abasto Social de Leche) in additional to the elimination of the Tortilla subsidy


Impact of progresa on poverty 2

Impact of PROGRESA on Poverty (2):

  • Figures 1 & 2: the effects of PROGRESA on hh income and poverty may not be adequately summarized by the size and incidence of the cash transfers

  • Econometric analysis based on individual and hh income in each round from a variety of sources: labor income, income from self-employment, other income (pensions, rent, and community profits) and government transfers (Ninos de Solidaridad, ININ, PROBECAT, PET, PROCAMPO) +PROGRESA Cash Transfers (from program admin records)


The impact evaluation of the progresa oportunidades program of mexico

Figure 1 – Mean household income from children

Among beneficiary households with children aged 8-17

Panel a

Mean household TOTAL income from Children

(excluding PROGRESA cash transfer)


The impact evaluation of the progresa oportunidades program of mexico

Figure 1 – Mean household income from children

Among beneficiary households with children aged 8-17

Panel b

Mean household LABOR income from Children


The impact evaluation of the progresa oportunidades program of mexico

Figure 1 – Mean household income from children

Among beneficiary households with children aged 8-17

Panel c

Mean household Other Income from Children


The impact evaluation of the progresa oportunidades program of mexico

Figure 2


The impact evaluation of the progresa oportunidades program of mexico

Impact on Poverty

where


Table 4 results 1

Table 4 results: (1)

  • PROGRESA had a significant impact in reducing poverty between November 1997 and November 1999.

    • E.g. using the 50th percentile of the value of consumption per capita as a poverty line, suggests that the headcount poverty rate declined by around 4.88% between November 1997 and November 1998 and by 18.11% in the November 1999 in treatment areas (using as base the 55.44% headcount poverty rate in treatment localities in November 1997).

    • Over the same period, and using as base the corresponding value of the poverty gap and squared poverty gap indices in treatment areas in November 1997, the poverty gap measure declined by 37.40%, and the severity of poverty measure (squared poverty gap) declined by 47.42%.

    • The higher impacts of the program in reducing poverty over time are consistent with the findings of Gertler et al. (2006), who demonstrated that rural households increased their investments in microenterprises and agricultural activities which, improved the ability of households to generate income.


Table 4 results 2

Table 4 results: (2)

  • Estimates are remarkably in line with the estimates obtained using ex-ante simulations.

    • E.g. simulations based on the predicted consumption of each household in the evaluation sample in November 1997 and adding the maximum amount of PROGRESA cash transfers an eligible household could receive assuming full compliance with the program’s requirements (see Skoufias et al., 2001).


Table 4 results 3

Table 4 results: (3)

  • The poverty reduction effects are stronger for the poverty gap and severity of poverty measures, which put greater weight on the poorest of the poor, and our evidence suggests that these estimated poverty effects are robust to the choice of different poverty lines.

  • Figure 3 and Appendix A


Concluding remarks

Concluding Remarks

  • PROGRESA does not have any significant effect on adult labor supply choices.

  • There is not much evidence to support the hypothesis that PROGRESA beneficiaries use their transfers to “buy” more leisure.

  • Results are reinforced further by the result that PROGRESA leads to a substantial reduction in current poverty.


The contributions of program evaluation 1

The Contributions of Program Evaluation-1

  • Program continued and improved

    • Fox administration (begun in 2001) kept and expanded the program

    • Early operations reports in PROGRESA identified implementation issues to be analyzed further (food supplements, intra-household conflict, targeting views)

    • Decision to maintain household targeting in PROGRESA expansion, but to add “self-selection” to administrative selection in urban areas


The contributions of program evaluation 2

The Contributions of Program Evaluation-2

  • Program Design improved: Program expanded to urban areas

    • Benefits extended to Preparatoria Secondary level

    • Jovenes con Oportunidades- aims to create income generating opportunities for poor households through preferential access to microcredit, housing improvements, adult education and access to social/health insurance.


Critical issues to be resolved on cct

Critical Issues to be Resolved on CCT

  • Do CCT programs break the intergenerational transmission of poverty? Need long time-series

  • What is the minimal CCT that may be paid?

    • Oportunidades: size of transfer based on opportunity cost of children (child wage/value of children’s contribution to family…)


Critical issues to be resolved

Critical Issues to be Resolved:

  • Impact on Children’s Achievement & Learning?

    • Do CCT increase achievement or induce teachers to lower grade-passing standards?

  • Teacher & health worker incentives

  • Quality of Services


Critical issues to be resolved1

Critical Issues to be Resolved:

  • Do CCT generate Program/Welfare Dependency?

    • So far no negative incentive effects on adultwork

    • Transfers & Income generation

  • Exit Rule?

    • Lack built-in flexibility to expand coverage to households falling below poverty during crisis


Final issue

Final Issue

  • Long-Run Sustainability of Program Budget & Political Economy of Program Support


The impact evaluation of the progresa oportunidades program of mexico

Thank you


Evaluation results targeting

Evaluation Results-Targeting

  • Geographic targeting of the program in rural areas is good

  • Method of selecting poor households within localities is generally accurate (undercoverage of 7% )

  • PROGRESA’s targeting decreases the poverty gap P(1) by 30% and the severity of poverty P(2) by 45%


Education

Education

  • Are more children attending school because of PROGRESA?

  • How much can schooling be expected to increase?

  • Are there more cost effective ways of bringing children to school?

  • Does PROGRESA have more impact in certain grades?

  • Any effects on drop-out rates, grade progression, repetition, reentry?


Evaluation results education

Evaluation Results: Education

  • PROGRESA has a positive effect the school attendance of both boys and girls in primary and secondary school

    • Boys in secondary: increase by 8 %

    • Girls in secondary: increase by 14%

  • Negative impact on children’s labor market participation (especially boys)

  • No observed increase in the attendance rate (frequency) of children in school.

  • PROGRESA increases overall educational attainment by 10% (and 8% higher earnings)


Evaluation results education1

Evaluation Results: Education

  • Program effective in keeping children in school especially during the critical transition from primary to secondary

  • Less effective in bringing back to and keeping in school children who were out.

  • Earlier entry ages, less grade repetition, better grade progression

  • PROGRESA more cost-effective than increasing access to junior secondary education


Health

Health

  • Does it increase visits to public health clinics?

  • Does PROGRESA have an effect on child health?

  • On the health of adults?


Evaluation results health

Evaluation results: Health

  • Significant increase in visit rates in PROGRESA communities

    • Increasing in nutrition monitoring visits, immunization rates and prenatal care in 1st trimester (8% increase)

  • No substitution between private and public facilities


Evaluation results health1

Evaluation results: Health

  • 12% lower incidence of illness in children between ages 0-5.

  • Significantly positive effects on adult health


Nutrition

Nutrition

  • Does PROGRESA impact of child growth?

  • Household consumption and food diet?


Evaluation results nutrition

Evaluation results: Nutrition

  • Significant effect at increasing child growth (1cm higher growth) and reducing the probability of stunting among children 12-36 mo.

  • Household total consumption increases

  • PROGRESA households “eat better” (higher expenditures on fruits, vegetables, meats & animal products)


Cost analysis

Cost Analysis

  • Are the administrative costs of PROGRESA high?

  • What are the private costs associated with participation in the program?

  • What might be the indirect effects of the program on the national economy? (e.g. financing of the program)?


Evaluation results cost analysis

Evaluation Results: Cost Analysis

  • For every 100 pesos allocated to the program, 8.2 pesos are administration/program costs.

    • Very low compared to LICONSA (40 pesos per 100 pesos) and TORTIVALES (14 pesos per 100)

  • Targeting and conditioning of the program makeup 56% of program costs (4.6 pesos out of 8.2 pesos)

  • Private costs (3.8 pesos out of 8.2 pesos)


Evaluation results cost analysis1

Evaluation Results: Cost Analysis

  • Eliminating distortionary food subsidies and using funds to finance a program like PROGRESA leads to substantial welfare gains.


  • Login