1 / 13

MEASURING IMPACT

MEASURING IMPACT. Impact Evaluation Methods for Policy Makers.

mauve
Download Presentation

MEASURING IMPACT

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MEASURING IMPACT Impact Evaluation Methods for Policy Makers This material constitutes supporting material for the "Impact Evaluation in Practice" book. This additional material is made freely but please acknowledge its use as follows: Gertler, P. J.; Martinez, S., Premand, P., Rawlings, L. B. and Christel M. J. Vermeersch, 2010, Impact Evaluation in Practice: Ancillary Material, The World Bank, Washington DC (www.worldbank.org/ieinpractice). The content of this presentation reflects the views of the authors and not necessarily those of the World Bank.

  2. 1 Counterfactuals False Counterfactuals Before & After (Pre & Post) Enrolled & Not Enrolled (Apples & Oranges) Causal Inference

  3. 2 Randomized Assignment Randomized Offering/Promotion Discontinuity Design Difference-in-Differences Diff-in-Diff IE MethodsToolbox Matching P-Score matching

  4. Prospective/Retrospective Evaluation? Eligibility rules and criteria? Roll-out plan (pipeline)? Is the number of eligible units larger than available resources at a given point in time? Choosing your IE method(s) • Key information you will need for identifying the right method for your program: • Poverty targeting? • Geographic targeting? • Budget and capacity constraints? • Excess demand for program? • Etc.

  5. Best Design Have we controlled for everything? Is the result valid for everyone? Choosing your IE method(s) Choose the best possible design given the operational context: • Best comparison group you can find + least operational risk • Internal validity • Good comparison group • External validity • Local versus global treatment effect • Evaluation results apply to population we’re interested in

  6. Progresa Policy Recommendation? Note: If the effect is statistically significant at the 1% significance level, we label the estimated impact with 2 stars (**). If significant at 10% level, we label impact with +

  7. 2 Randomized Assignment Randomized Offering/Promotion Combinations of methods Discontinuity Design Difference-in-Differences Diff-in-Diff IE MethodsToolbox Matching P-Score matching

  8. Choosing your method

  9. The objective of impact evaluation is to estimate the causal effect or impact of a program on outcomes of interest. Remember “

  10. To estimate impact, we need to estimate the counterfactual. what would have happened in the absence of the program and use comparison or control groups. Remember “

  11. We have a toolbox with 5 methods to identify comparison groups. Remember “

  12. Choose the best evaluation method that is feasible in the program’s operational context. Remember “

More Related