1 / 18

Causal Inference

Causal Inference. David Evans Economist Africa Impact Evaluation Initiative. Goal. Evaluate the causal impact of a program or an intervention on some outcome How much did X move Y? How much did the new teacher contracts affect students performance? Not the same as correlation !

jimmyz
Download Presentation

Causal Inference

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Causal Inference David Evans Economist Africa Impact Evaluation Initiative

  2. Goal • Evaluate the causal impact of a program or an intervention on some outcome • How much did X move Y? • How much did the new teacher contracts affect students performance? • Not the same as correlation! • X and Y move together • Holding an umbrella & having wet shoes • Does the umbrella cause wet shoes? • Age and income

  3. Evaluation problem • Compare same individual with & without a program at the same point in time • BUT Never observe same individual with and without program at same point in time

  4. Solving the evaluation problem • Counterfactual: what would have happened without the program

  5. A little more rigorously • Estimate counterfactual • find a control or comparison group • Counterfactual Criteria • Treated & counterfactual groups have identical initial average characteristics • Only reason for the difference in outcomes is due to the intervention Mick Pick B- A+

  6. “Counterfeit” Counterfactual #1 • Before and after: • Same individual before the treatment • But what else is happening over time? Before the books

  7. Before and After: Food aid • Compare income before and after • Income after food aid is lower than income before food aid • Did the program fail? • Received aid because of income fall • Cannot separate (identify) effect of food aid from the shock that led to needing it

  8. Before & After:Cash transfers Income • Compare income before & after intervention Before & after counterfactual = B Estimated impact = B-A • Control for time varying factors True counterfactual = C True impact = A-C B-A is gives a false negative! Before After B Negative impact!! B A C Positive impact!! t-1 t Time Treatment (Transfers)

  9. “Counterfeit” Counterfactual #2 • Compare participants to non-participants: • Those who choose not to enroll in program, or • Those who were not offered the program • Problem: We can not determine why Mick participates in the program and Don does not • Why didn’t Don get books?

  10. Why participants and non-participants might differ? • People who claim unemployment benefits and people who do not? • People who go to the doctor and people who do not? The unemployed The sick

  11. Health program example • Treatment offered • Who signs up? • Those who are sick • Areas with epidemics • Have lower health status than those who do not sign up • Healthy people/communities are a poor estimate of counterfactual

  12. What's wrong? • Selection bias: People choose to participate for specific reasons • Many times reasons are directly related to the outcome of interest Mick gets better marks with books B- A+ But what if Mick’s received books because he was already getting better marks A+ B-

  13. HIV: Effect of VCT on risk behaviors • What what if Mick went for VCT because he knows he engages in high-risk behaviors? (Even after, still higher than Don!) • Cannot separately identify impact of the program from these other factors/reasons No VCT VCT High risk behaviors Low risk behaviors

  14. Possible Solutions… • Guarantee comparability of treatment and control groups • ONLY remaining difference is intervention • How? • Experimental design/randomization • Non-experimental (difference in difference, matching, regression discontinuity, instrumental variables, etc)

  15. These solutions all involve… • EITHER Randomization • Give everyone an equal chance of being in control or treatment groups • Ensures that participants and non-participants will be similar on most characteristics* • Only difference is the intervention • OR Transparent & observable criteria for assignment into the program

  16. Finding controls: opportunities • Budget constraints: • Eligible who get it = potential treatments • Eligible who do not = potential controls • Roll-out capacity: • Those who go first = potential treatments • Those who go later = potential controls

  17. Finding controls: ethical considerations • No need to delay benefits if rollout based on budget/capacity constraints • Equity: equally deserving populations deserve an equal chance of going first • Transparent & accountable: If based on criteria, then criteria should be measurable and public • Ethics binds after you know it works

  18. Conclusions • Seek to measure the causal effect of a program on some outcome • Need a counterfactual • Shun the counterfeits! • Before and after • Participants vs non-participants • Find a good control group • Would be the same in the absence of the intervention • Options • Randomize (coming soon!) • Use other methods with more assumptions (coming tomorrow!) THANK YOU

More Related