doing impact evaluations in argentina and chile n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Doing Impact Evaluations in Argentina and Chile: PowerPoint Presentation
Download Presentation
Doing Impact Evaluations in Argentina and Chile:

Loading in 2 Seconds...

play fullscreen
1 / 12

Doing Impact Evaluations in Argentina and Chile: - PowerPoint PPT Presentation


  • 130 Views
  • Uploaded on

Doing Impact Evaluations in Argentina and Chile:. Pragmatism, Opportunities and Partnership. Country Context . Argentina Weaknesses in Survey Data (and worsening) Highly Politicized, Particularly Relations between Central and Lower Levels of Government Lack of institutional capacity Chile

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Doing Impact Evaluations in Argentina and Chile:' - calandra


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
doing impact evaluations in argentina and chile

Doing Impact Evaluations in Argentina and Chile:

Pragmatism, Opportunities and Partnership

country context
Country Context

Argentina

  • Weaknesses in Survey Data (and worsening)
  • Highly Politicized, Particularly Relations between Central and Lower Levels of Government
  • Lack of institutional capacity

Chile

  • Evaluation Culture Led by Ministry of Economy
  • Requirement for evaluation in Program law
impact evaluations 1997 2006
Impact Evaluations 1997-2006

Argentina

  • Trabajar (workfare) Program
  • Workfare Participation and Exit
  • Private Employment Subsidy
  • Heads of Household Project
  • Grant-financed Productive Activities
  • Future (?) Lifelong Learning and Training and Employment Services

Chile

  • Chile Solidario
motivations
Motivations
  • Serious link with possible follow-up operation (Trabajar) – role of Bank management
  • Risk mitigation measure (Heads of Household/grants for productive activities)
  • Government/Bank shared technical interest (private employment subsidy, study of workfare leavers, Chile Solidario)
pragmatic design
Pragmatic Design
  • Used existing surveys (Social Survey, Labor Force Survey, CASEN)
  • Use of Administrative Data
  • Partnered with local institutions (Ministry of Labor, SIEMPRO, INDEC, MIDEPLAN)
  • Partnered with DEC staff (Bank supervision budget and other resources)
  • Government financed out of loan funds
  • Cost-effective impact evaluations
pragmatic design continued
Pragmatic Design continued
  • Argentina productive projects: 3rd best design, rigorously implemented
  • Chile: Bank came in after program started, based on how program was implemented, recovery from last minute change on sample
  • Don’t give in on key items – sample size, innovation (psycho-social questions), quality of technical advice on evaluation
  • Random Assignment sometimes possible
timing critical
Timing Critical
  • Trabajar: Board approval (6/97)

Prel. Evaluation results (5/98)

  • HoH: Program start (4/2002)

Prel. Evaluation results (7/2003)

Chile

Solidario: Information for New Government

devil is in the details
Devil is in the Details
  • Be ready for close supervision and lots of nitty-gritty work
  • Carefully monitor implementation (samples, field work, questionnaires) -even then things will go wrong
  • Data quality issues
  • Be ready to explain design and methodology many times
  • Not a one-shot deal
pay off of sustained partnership
Pay-off of Sustained Partnership
  • Building of Relationships (INDEC, DIPRES- Budget Office in Chile)
  • Building of Relationships with Bank staff (DEC)
  • Building of Understanding of Methodologies (Ministry of Labor, DIPRES)
  • Aiming to make it “second nature”
link with operations of programs
Link with Operations of Programs
  • What are the key questions?
  • How is program being implemented?
  • What administrative data is available?
  • Knowledge/cooperation of operational staff
  • Important for following up on results
  • Add value to evaluation – heterogeneity of impacts
  • Best partners for impact evaluation are program operators. They do not want everything to focus on methodology.
use of results
Use of Results
  • Trabajar – Justified follow-on operation
  • Head of Household – Program credibility,

provided information on implementation

  • Chile Solidario – too soon to tell
  • “Public goods” contribution
  • Spill-over effects (use of data by others)
but not a magic bullet
…but, Not a Magic Bullet
  • Decisions on programs based on many factors, not just their performance
  • Not everyone convinced by evidence-based data vs. anecdotes or ideology or initial views
  • Need for better dissemination, particularly in-country, and making results understandable
  • Bank value-added, including for middle-income countries
  • We need to work more broadly on sustainability issues