Causality in Social Work Research via Instrumental Variable Estimation

1. Causality in Social Work Research via Instrumental Variable Estimation Roderick A. Rose Senior Research Associate Carolina Institute for Public Policy The University of North Carolina at Chapel Hill

2. Summary • Rationale: endogeneity & causality • Instrumental variable estimation • The exclusion restriction: justification and identification; logic; example • Recommendations

3. Rationale:Endogeneity & Causality Endogeneity is pervasive throughout non-experimental research.

4. What is Endogeneity? This is a Directed Acyclic Graph, or DAG. Causality flows in the direction of the arrow. This DAG represents what we might want to show (3 examples), representing a causal relationship between X and Y. X Y Rental housing assistance Employment Career-relevant instruction Math achievement Health insurance Health at time t

5. What is Endogeneity? Endogeneity occurs as an undesired relationship between variables in this causal relationship and other variables not accounted for in the model. C Education Level X Y Rental housing assistance Employment Eliminating endogeneity requires removing one of the arrows.

6. What is Endogeneity? The arrows corresponding to the confounder may go in any direction; endogeneity has no regard for the direction of the arrow. C Teaching quality X Y Career-relevant instruction Math achievement Note that any relationships not modeled occur through the error term. This is the definition of endogeneity: a correlation between an effect an the error.

7. What is Endogeneity? When simultaneity or reverse causality are needed, they are represented in a DAG by lagged versions of variables. Health at time t-1 C b < 0 X Y b > 0 Public Health insurance Health at time t

8. A Common Risk in Social Work Research? When simultaneity occurs in the context of cross-sectional data collection (i.e., one time point), we cannot be sure which effect we are observing. X Y b > 0 or b < 0 Public Health insurance Health Note that this is not technically a DAG (because of the two headed arrow).

9. Description vs. Causality It is entirely appropriate to use regression methods to describe relationships among variables without inferring causality. Race/ethnicity C X Y Public Health insurance Health at time t This curved line represents an unspecified relationship between two variables; implicit in this is that the relationship occurs through or because of other variables.

10. Instrumental Variable Estimation Notation X = EV = endogenous variable Y = DV = dependent variable Z = IV = instrumental variable or instrument

11. The Instrumental Variable or IV In large samples, the instrument is a variable Z that is highly correlated with X and uncorrelated with the error. Health at time t-1 Not measured (x-sect) C Z X Y Public Health insurance Health at time t The IV must effectively randomize participants to conditions of X. Like random assignment, it cannot be confounded by any C.

12. IV Estimator using Covariance (Diagram)Circles represent variance; Source: Kennedy, 2003. Y = DV error in Y Used to estimate βOLS Note the loss of efficiency. Used to estimate βIV X = Endog Var Z = InstrVar Note overlap with error No overlap with error No overlap with Y outside of X

13. IV Estimator (Diagram)Circles represent variance; Source: Kennedy, 2003. Y = DV Predict X’s covariance with Y Predicted X X = Endog Var Z = InstrVar

14. IV Estimator (Diagram)Circles represent variance; Source: Kennedy, 2003. Y = DV r X X = Endog Var Z = InstrVar

15. Common Instruments • “Natural” phenomena including births, weather events, geographical boundaries. Katrina is a good example. • Randomized treatments. These are a particular class of IVE that are used when doing mediation or dosage type models (local average treatment effects). This is also (under certain assumptions) the Bloom Estimator (Bloom, 1984): Divide the ITT of the outcome by the ITT of the dosage. • Functions and aggregates of observed characteristics. A person’s ethnic background will not satisfy exclusion, but membership in a “cluster” defined by ethnicity, age, income, job and other characteristics might. See Cutler & Gruber, 1996. • Taking the cluster idea one step further, multilevel data yield “natural” clusters that can be used, under certain assumptions, to produce useful instruments. This is often referred to as a “Wald instrument” because it accords with Wald’s hypothesis. See Cobb-Clark & Crossley, 2003.

16. The Wald Estimator and IVE • “If Z is a dummy variable indicating membership in one of two groups, then zy′/n captures the difference in the average outcomes of the two groups and zd′/n captures the difference in the treatment intensity across the two groups. Thus the IV estimator simply scales the difference in average outcomes by the difference in treatment intensities, as our intuition suggests.” -Cobb-Clark and Crossley, 2003. page 504.

17. The Exclusion Restriction: Justification and Identification

18. Controversy Surrounding IVs • The most problematic aspect of IVE: search for a variable satisfying the requirements of no correlation with error and high correlation with endogenous variable. • Even some economists are skeptical of the value of certain approaches taken to estimating IVs: Heckman (1990; 1997). • Heckman even questions random assignment as an instrument! • Inadequate attention to ensuring exclusion is a frequent problem. • This is changing particularly as researchers outside of economics begin to see the value of the method.

19. Can Exclusion be Proven? The (flawed) logic of a test for exclusion is that if Z is associated with Y after conditioning on X, then it is not an IV, because that means it must operate through C (or any other unmeasured variable). Health at time t-1 Not measured (x-sect) C Z X Y Food stamp receipt Public Health insurance Health at time t The problem is that this test always fails, because conditioning on X creates an artifactual association between Z and C. Exclusion must be “proven” by inductive methods only.

20. Can Exclusion be Proven? 1 Propensity to receive food stamps 0 Health at time t-1 Green: does not have public health insurance pink: has public health insurance Refer to: Morgan & Winship, 2007; p. 67.

21. Exclusion Logic: Example • A school district recently implemented a policy change, promoting students to middle school in 7th grade rather than 6th. • You are an evaluator given the task of determining whether this policy is beneficial to students, based on 7th grade exams. • Promotion is endogenous because it is determined partly by performance on end-of-grade exams. • Having been hired after implementation, randomization is not an option. • If a valid IV can be identified, you might have a causal estimate. • Birth date is random, right? (Right?)

22. Exclusion Logic: Example of Potential IV Aug 31 Sept 1 Dec 31 Jan 1 • Blue cohort: advances in 6th grade. • Green cohort: advances in 7th grade. • Students in different cohorts nearest the star are most alike, and thus less subjected to random differences. • In calendar Year B, birth date determines cohort membership. • Correlation between birth date and cohort determined by dispersion over year-long period before/after star. •  Quality of IV (correlation with cohort; influence of error) determined by the distance of each student’s birth from the star. Yr A Yr B Yr C

23. Discussion:Heintze, Berger, Nadich & Meyers (2006) • Question: Is housing assistance directly associated with employment among low-income single mother households? • Endogeneity problem? • “The instruments used here are lagged measures of state-level housing assistance generosity. They are constructed from HUD data (HUD 1998). The lagged measure of housing assistance generosity for the 1997 sample is the ratio of the number of federally subsidized units to the number of poor households in the state in 1993.” (p 649) • How is exclusion assumed here? • How might exclusion fail? • Did the authors sufficient examine exclusion? • What alternatives might exist?

24. Recommendations

25. Recommendations: Endogeneity • Never include a known endogenous variable as an intermediary between your focal causal variable and your outcome (e.g., do not include dosage when modeling treatment). • The seeming exception to this is mediation, though in this case the mediator is the focal causal variable. • There are many methods for dealing with endogeneity, including fixed effects, interrupted time series, PSM, regression discontinuity; some of these remain relatively untapped by social work researchers.

26. Recommendations: IVE • Knowing what an IV is and how it works will be important for social work evaluators and researchers as the disciplines converge on scarce grant funds for research projects. • When doing mediation or dosage (efficacy; i.e., not intent-to-treat) analysis, IVE is a potentially valuable tool, particularly when the treatment (as an antecedent to dosage) is randomly assigned. • Look for next paper on local average treatment effects. • This method also appropriate when ethics prevents randomization directly to treatment. • Think about what instruments might be useful in your work; look for examples of instruments by other researchers in the literature. Be critical; a lousy instrument does not promote good science.

27. Recommended Reading • Morgan, S. L. & Winship, C. (2007). Counterfactuals and Causal Inference: Methods and Principals for Social Research. Cambridge University Press. • There are about a dozen articles in the social work literature that use or refer to IVE. These are listed in the appendix of the Stone article on endogeneity. • The papers I’ve referred to: • Cutler, D.M., and J. Gruber. 1996. The Effect of Expanding the Medicaid Program on Public Insurance, Private Insurance, and Redistribution. American Economic Review: Papers and Pro-ceedings 86(2): 368–373. • Cobb-Clark, D. A. & Crossley, T. (2003). Econometrics for evaluations: An introduction to recent developments. The Economic Record, 79(247), 491-511. • A three part series in Social Science and Medicine by myself & Susan Parish & colleagues, and Michael Foster: • Rose, R. A., Parish, S. L., Yoo, J. (2010). Observation, correlation and policy analysis: A response to Foster. Social Science & Medicine, 70, 1274-1276. • Foster, E. M. (2010). A commentary on Rose et al. Social Science & Medicine, 70,1271-1273. • Rose, R. A., Parish, S. L., Yoo, J., Grady, M.D., Andrews, M. E., Powell, S. E., & Hicks-Sangster, T. K. (2010). Medicaid suppresses racial disparities in health care among children with special health care needs: Evidence from a multi-level mediation modeling analysis. Social Science & Medicine, 70, 1263-1270.

28. SSWR in Washington DC, Jan 2012 • I will be presenting a paper using LATE via IVE methods in a symposium for CareerStart that has several papers by myself, Mike Woolley and Jay Unick. • While my time for explaining the intricacies of LATE will be limited, I will have (by this time) briefs and probably a short report. • The paper I am presenting will constitute one of my dissertation papers.

29. Closing Comments & Questions