Hammers and Saws for the Improvement of Education Research

1. Hammers and Saws for the Improvement of Education Research Margaret Eisenhart University of Colorado IEREA Luncheon Address December 10, 2004

2. A full version of the paper on which this presentation was based will appear in the Summer 2005 issue of Educational Theory.

3. Introduction Quarrels over which method represents the ‘gold standard’ make no more sense than arguing about whether hammers are superior to saws. The choice depends on whether you want to drive a nail or cut a board. (Schorr & Yankelovich, 2000, The Los Angeles Times)

4. Overview of Talk • Importance of Asking Questions about Causation in Education Research Today • Fact that Answering Questions about Causation with Research Requires More than Experiments (hammers + saws) • Examples • Arguments • The Need for More Integrated Research Approaches

5. Causation 2 Approaches to Causation • Variance/regularity theory: Q: Does change in x cause change in y? To Answer: Use experimental methods • Process/realist theory: Q: How does x affect y? To A: Use ethnographies, case studies, discourse analyses, narrative analyses, etc.

6. Experimental Studies of Causation • In the past 25 years, few have been done; more could be done. • When possible and ethical, experiments yield useful results about relationships among variables. • Experiments do not answer all important research questions about causation, but they do answer some.

7. Main Reference Cited: Valentine, J. & Cooper, H. (in press). Scaling the quality of causal research in education. To appear in Experimental methods of educational interventions. G. Phye, D. Robinson & J. Levin, eds. San Diego: Academic Press.

8. Scientifically-Based Research • Scientifically-based research in education seeks to answer 3 types of research questions: 1. “Descriptive” questions 2. “Causal effects” questions 3. “Explanatory” questions • Only #2 is answerable with experiments. • Yet all 3 must be answered for a good understanding of causation.

9. Descriptive Q: What is happening? • Causal effects Q: Is there a systematic relationship between x and y? • Explanatory Q: How or why does x cause y?

10. Aristotle’s 4 “Causes of a Thing” • Material cause—that out of which a thing is made • Formal cause—that into which a thing is made • Efficient cause—that which makes the thing • Final cause—that for which the thing is made

11. Main References Cited Holland, P. (2004). Evidence for causal inference in education. Paper presented at AERA, April. Maxwell, J. (1996). Qualitative research design. Thousand Oaks, CA: Sage Publications. National Research Council (2002). Scientific research in education. Washington, DC: National Academy Press.

12. 2 Process Views of Causation • Realist/Intentional View • Practical View

13. Realist/Intentional View • Human behavior, human intentions, and social interactions cause human events. • These phenomena lose their causal power when experimentally manipulated. • Best research methods are direct observation, participation, and questioning.

14. Main References Cited Howe, K. (2004). A critique of experimentalism. Qualitative Inquiry, 10 (1). Maxwell, J. (2004). Causal explanation, qualitative research, and scientific inquiry in education. Educational Researcher, 33 (2).

15. Practical View • Human events are caused by the circumstances that precede them in time. • These phenomena lose meaning when taken out of the context of history. • Best research methods are historical case studies of power dynamics among groups over time.

16. Reference Cited Flyvbjerg, B. (2001). Making social science matter. Cambridge: Cambridge University Press.

17. Conclusion • All 4 views of causation are relevant to educational phenomena. • Multiple approaches and tools--hammers, saws, and more--are needed to develop good understandings of causation that can advance the field of education research.