Matching Methods - 2

1. Matching Methods - 2 Methods of Economic Investigation Lecture 12

2. Last Time • Ways to define a ‘control group’ if you don’t have an experiment • Difference-in-Differences • Assume: Fixed Differences over time • Attribute any change in trend to treatment • Propensity Score Matching • Assume: Treatment, conditional on observables, is as if randomly assigned • Attribute any difference in outcomes to treatment

3. Choices when doing p-score matching • Sample with or without replacement • One-to-one or one-to-many matching • How many observations to use for a match • What criteria to just how close is “close enough”

4. How close is close enough? • No “right” answer in these choices—will depend heavily on sample issues • How deep is the common support (i.e. are there lots of people in both control and treatment group at all the p-score values • Should all be the same asymptotically but in finite samples (which is everything) may differ

5. Tradeoffs in different methods Source: Caliendo and Kopeinig, 2005

6. How to estimate a p-score • Typically use a logit • Specific, useful functional form for estimating “discrete choice” models • You haven’t learned these yet but you will • For now, think of running a regular OLS regression where the outcome is 1 if you got the treatment and zero if you didn’t • Take the E[T | X] and that’s your propensity score

7. The Treatment Effect J is comparison group with |J| is the number of comparison group units matched to i N is the treatment group and |N| is the size of the treatment group CIA holds and sufficient region of of common support Difference in outcome between treated individual i and weighted comparison group J, with weight generated by the p-score distribution in the common support region

8. General Procedure • Run Regression: • Dependent variable: T=1, if participate; T = 0, otherwise. • Choose appropriate conditioning variables, X • Obtain propensity score: predicted probability (p) • 1-to-1 match • estimate difference in outcomes for each pair • Take average difference as treatment effect • 1-to-n Match • Nearest neighbor matching • Caliper matching • Nonparametric/kernel matching Multivariate analysis based on new sample

9. Standard Errors • Problem: Estimated variance of treatment effect should include additional variance from estimating p • Typically people “bootstrap” which is a non-parametric form of estimating your coefficients over and over until you get a distribution of those coefficients—use the variance from that • Will do this in a few weeks

10. Some concerns about Matching • Data intensive in propensity score estimation • May reduce dimensionality of treatment effect estimation but still need enough of a sample to estimate propensity score over common support • Need LOTS of X’s for this to be believable • Inflexible in how p-score is related to treatment • Worry about heterogeneity • Bias terms much more difficult to sign (non-linear p-score bias)

11. Matching + Diff-in-Diff • Worry that unobservables causing selection because matching on X not sufficient • Can combine this with difference and difference estimates • Take control group J for each individual i • Estimate difference before treatment • If the groups are truly ‘as if’ random should be zero • If it’s not zero: can assume fixed differences over time and take before after difference in treatment and control groups

12. Bottom Line… • Matching Methods used to replicate experimental methods • Need to believe independence, conditional on X’s • If matching assumption is right, can estimate the TOT without worrying about selection bias