An Intro to Qualitative Methods

1. An Intro to Qualitative Methods Elements of this lecture have been shamelessly borrowed from: Mahoney & Goertz, “ A Tale of Two Cultures: Contrasting Quantitative and Qualitative Research” Political Analysis vol. 14 (2006): 227-249

2. Contents • Common differences between qualitative and quantitative research methods • Strategies of case selection • “Most similar” and “Most different” comparisons • “Hard” and “Easy” cases • Testing for causal mechanisms • Limitations of qualitative research

3. Caveat • “Qualitative methods,” like “quantitative methods,” is an umbrella term which encompasses a wide variety of research methods including, but not limited to: ethnography, comparative historical analysis, linguistic analysis, & interpretivist narratives. • THIS lecture focuses specifically on the method of case-driven comparisons, which assumes that causal relationships exist and can be observed.

4. Qualitative and Quantitative Approaches to explanation • Qualitative: Look for the causes of a phenomenon in a single or select group of cases • What are the causes of the rise of Fascism in pre-WWII Germany & Italy? What are the causes of authoritarian breakdown in Indonesia? • In qualitative analysis, the aim is to “work backwards” from a known outcome (e.g. “War occurred between these two states”) and identify the causes of that outcome (e.g. “What caused the war to happen?). Hence: The search for “causes of effects”

5. Qualitative and Quantitative Approaches to explanation • Quantitative: Look for the average effects of one or more potential factors over a wide population of cases. • What is the effect of democratization on economic growth? What is the effect of natural resources on civil war length? • In quantitative analysis, the aim is to “work forwards” by taking a group of potential causes (e.g. “Economic growth, regime type, military spending”) and observing their net effects on an outcome (e.g. “What are the effects of these factors on the likelihood of war breaking out?”). We don’t necessarily know the outcomes in advance. Hence: the search for “effects of causes”

6. Qualitative and Quantitative Implications • In qualitative research, answers should identify the specific causal process that led to an outcome for a single or select group of cases • The collapse of the Labor Party led to elite defections to the right which then led to the rise of the Conservative Party in Great Britain • In quantitative research, answers should aim to identify generalizable causal relationships that occur across as many cases as possible • In-fighting among left-wing parties can predict the electoral victories of right-wing parties in a majority of cases.

7. Qualitative and Quantitative Scope and Generalization • Qualitative: Adopt narrow scope conditions that limit the population of your cases in order to avoid over-generalizations. • Ex: Skocpol (1979) – Study of social revolutions among agrarian-bureaucratic states that have not been colonized. Population of cases: Russia, China, and France.

8. Qualitative and Quantitative Scope and Generalization • Quantitative: Adopt broader scope conditions that increase the number of observations and enhance the generalizability of findings. • Ex: Singer & Small (1976) - Countries in which at least 10 percent of all adults are eligible to vote are considered “democracies”

9. Qualitative and Quantitative Implications • The dangers of causal heterogeneity: Causal relationships do not remain constant across all cases (e.g. democracies are positively correlated with economic growth in rich countries but negatively correlated with economic growth in poor countries) • Qualitative research reduces the chances of mis-specifying causal relationships by narrowing the population of cases, which comes at a cost to generalizability. • Quantitative findings are more stable and generalizable when explaining systemic trends but are more vulnerable to mis-specifying causal relationships.

10. Qualitative and Quantitative Weighting Cases • Qualitative: Focus on exceptional or important cases. • Certain cases/observations are inherently more “important” than other cases. • Theories of authoritarian breakdown in Communist states should be able to explain the downfall of the Soviet Union. Theories of the rise of Fascism in Europe should be able to account for Nazi Germany. • “Exceptional cases” that buck existing trends should not be dismissed as being part of the “error term”; these cases still warrant explaining (i.e. “Empirical puzzles”)

11. Qualitative and Quantitative Weighting Cases • Quantitative: All cases/observations are equally weighted. • The United States is no more important than Moldova when treated as a singular observation. • A study that can (improbably) explain the causes of democratization in every country in the world except for in the United States, Great Britain, France, and Germany would still be phenomenally useful given that its findings are generalizable to the overwhelming majority of cases.

12. Qualitative and Quantitative Implications • In qualitative research, failing to explain a single “important” case (that meets all relevant scope conditions) can seriously undermine the credibility of an argument. • This is also a product of having narrow scope conditions • Ex: Skocpol and the Iranian Revolution • In quantitative research, one or several “exceptional” cases can be considered “unsystematic” and relegated to the error term. • Ex: Democratic Peace and the Syracuse-Athens War

13. Qualitative and Quantitative Conceptualization and Operationalization • Qualitative: Clearly define concepts so as to avoid measurement error and conceptual over-stretching. • “Know what you’re talking about before trying to measure it” • Ex: What is State Capacity? What is Culture? What is Nationalism?

14. Qualitative and Quantitative Conceptualization and Operationalization • Quantitative: Clearly operationalize your variables by identifying appropriate indicators. • “If a concept has no observable indicators, then it is not a useful concept for causal inference” • Ex: State capacity can be adequately measured by observing direct taxation as a share of overall GDP. Culture can be adequately measured by observing shared knowledge of three national myths. Nationalism can be adequately measured by asking subjects to give an ordinal ranking of national pride.

15. Quantitative and Qualitative Implications • For qualitative researchers, measurement begins at concept formation. • Flawed or inappropriate measurements require re-adjusting the concept • Ex: A measurement term more accurately describes federalist systems than parliamentary systems; the project is no longer about parliamentary systems • For quantitative researchers, measurement begins with the search for observable indicators. • Flawed measurements require a better indicator • Ex: Direct taxation as a share of GDP is a better measurement of state capacity than GDP per capita; the project is still about state capacity

16. Case Selection • In qualitative research: cases are generally selected on the dependent variable with a preference for “positive” cases and the occasional “negative” case. “Null” cases are ignored. • In quantitative research: cases are randomly selected on the independent variables and all cases are analyzed.

17. Case Selection Most Similar Cases; JSM’s Method of Difference • Two cases with opposite outcomes that are nonetheless similar in almost every circumstance except for one. • Ex: Country A fought a war; Country B did not Country A = Low Econ Growth, Democracy, Low Pop Country B = High Econ Growth, Democracy, Low Pop Low levels of economic growth are a necessary condition for explaining the onset of war; does NOT indicate that low econ growth is a sufficient condition for explaining war onset.

18. Case Selection Most Different Cases; JSM’s Method of Agreement • Two cases with the same outcome that are dissimilar in almost every circumstance except for one. • Ex: Country A and Country B both fought wars Country A – Low Econ Growth, Autocracy, High Pop Country B – Low Econ Growth, Democracy, Low Pop Low Economic growth is (once again) a necessary condition for war onset.

19. Case Selection • Most Similar and Most Different Cases are often used to compare two different cases (i.e. across-case variation), but can also be used for single case studies by comparing the same case at different points in time (i.e. over-time variation). • U.S. under Bush Presidency vs U.S. under Obama Presidency; Korea pre-Japanese Occupation vs Korea under Japanese Occupation; Colonial India vs Post-colonial India • Looking at temporal (over-time) variation (usually) allows for the control of factors such as culture, demographics, geography, and other circumstances that would normally vary across two different states/entities.

20. Case Selection Weaknesses of Most Similar and Most Different Case Comparisons • Not actually a very good tool for identifying necessary causes • Consider two states that both fought wars State A: High Ice Cream Sales, Temperate, Woods State B: High Ice Cream Sales, Arid, Plains High ice cream sales are a necessary condition for explaining the onset of war?????

21. Case Selection Weaknesses of Most Similar and Most Different Case Comparisons • Difficult to control for every single possible confounding factor using a few case comparisons • Common sense only goes so far in ruling out unlikely factors like ice cream sales; what about terrain, farmland distribution, or factory distribution? • Then why use comparative cases? • Carefully selected case comparisons can help rule out the most common alternative explanations. • Ex: Literature links high economic growth to higher tax rates; select case comparisons that show high economic growth is (by itself) unable to account for higher tax rates.

22. Case Selection Easy and Hard cases • Easy case: a case where we would be most likely to expect our arguments to succeed (i.e. fewer confounding factors) • Failure to explain an easy case can be fatal to an argument • A theory that explains voter trends among stable, liberal democracies should be able to explain outcomes in Canada • Hard case: a case where we would be most likely to expect our arguments to fail (i.e. more confounding factors) • Successfully explaining a hard case can substantially strengthen an argument • A theory that explains voter trends among stable, liberal democracies would have a harder time explaining outcomes in India, where ethnic polarization is more likely to play a confounding role

23. Testing for Causal Mechanisms • Ultimately, comparative case selections are useful when controlling for alternative arguments, but provide only facile evidence for identifying causes of effects. • This is where testing for mechanisms (or process-tracing) comes in!

24. Testing for Causal Mechanisms • Most qualitative projects identify a causal process that links a set of causes to an outcome. In other words, what is your causal story? • “Goldilocks walked into a house and then got eaten by bears” is not a satisfying causal story. Plenty of people walk into houses and don’t get eaten by bears. What were the steps in the story that linked an antecedent condition (i.e. walking into a house) with the outcome (i.e. devouring by bears)? • Goldilocks walked into a house Goldilocks broke little bear’s property Goldilocks ate little bear’s porridge Goldilocks slept in little bear’s bed Bear parents observed little bear’s thrice-inflicted pain Maternal instincts of bear parents kick in Bear parents devour Goldilocks

25. Testing for Causal Mechanisms Democratic States  ?????  No War Dem. States Share Liberal Values Mutual Trust No War Dem. States Inst. Transparency Mutual Trust No War Dem. States Anti-war electorate No War Dem. States Economic growth Strong military  No War Dem. States Economic interdependence No War Dem. States Embeddedness in IGO’s No War Dem. States More military allies  No War Dem. States Prioritize autocrat enemies No War

26. Testing for Causal Mechanisms Moore (1966) Weak Middle Class + Divided Peasantry + Powerful Landed Elite = FASCISM Powerful Landed Elites More resources/votes for Fascist candidates Weak Middle Class + Divided Peasantry Weak opp. to Fascist cand. Confirming evidence? Falsifying evidence?

27. Testing for Causal Mechanisms • Identify the potential causal mechanisms that link your IV’s to your DV’s • Break down these mechanisms into a series of “steps” or “processes” • Identify observable indicators/implications that would confirm or falsify each step or process • Look for those observable implications in the empirical record

28. Limitations • Better suited to explaining extreme or clear categorical variations in the outcome • Qual: War vs No War; Revolution vs No Revolution; Coups vs No Coups; Maneuver vs Attrition vs Punishment strategies • Quant: 5% growth in GDP from 1991-1992; 7% decrease in voter satisfaction from 1996-2000 • Difficulty in weighting mechanisms that all predict the same outcome • Mechanisms can still be falsified, but if confirming evidence is found for two or more mechanisms, then problems of weighting come in. • External validity • More of a concern when explaining cases that meet similar scope conditions.

29. Limitations • Not all political phenomena are easily observable • Cultural context • Clifford Geertz’s “Thick vs Thin Description” • A conspiratorial eye-wink • An inadvertent eye-twitch • Eye-wink  A contraction of the eye muscle • Eye-twitch  A contraction of the eye muscle

30. Go Home