The Social Scientific Method

1. The Social Scientific Method An Introduction to Social Science Research Methodology

2. The Scientific Method Steps in the Scientific Method Make an observation Formulate a hypothesis about that observation Make predictions based on the hypothesis Test the predictions of the hypothesis using experiments How is this translated for the social sciences and humanities?

3. Research Design in the Social Sciences & Humanities Translating the scientific method for the social sciences & humanities: Make an observation  Make an observation Formulate a hypothesis about that observation  Formulate a hypothesis about that observation Make predictions based on the hypothesis  Make predictions (i.e., observable implications) about what we should expect if the hypothesis is correct Test the predictions of the hypothesis using experiments  Find, gather, and use data to test the observable implications of the hypothesis

4. The Scientific Method →The Social Scientific Method Make an observation  Make an observation • Develop a research question based on a puzzle or unexplored topic • What are puzzles? • Observations that existing theories do not explain • Observations that contradict existing theories • What are unexplored topics? • Phenomena that are not addressed by existing theories • Phenomena that have not been studied in a rigorous fashion • Puzzles and unexplored topics can overlap, so don’t think of these as hard and fast categories • The research question must be answerable! (Karl Popper)

5. The Scientific Method →The Social Scientific Method Formulate a hypothesis about that observation  Formulate a hypothesis about that observation • The hypothesis is a formal statement of why the observation matters • The observed phenomenon is caused by something • The observed phenomenon causes something else • The hypothesis should include a causal mechanism that describes the cause or effects of the observation • X has an effect on Y • Direct effects • Indirect effects • The causal mechanism is about how X affects Y • Embed the hypothesis in a theoretical framework

6. The Scientific Method →The Social Scientific Method • Make predictions based on the hypothesis  Make predictions and derive observable implications • Observable implications are predictions about what should be observed if the hypothesis is true • If my hypothesis is true, then we should observe . . . • Tied to causal mechanism • Ceteris paribus reasoning • Ideally they are “uniquely identified” • This means they are only true if the hypothesis is true • This is extremely rare; the important point is to have a statement about what you expect to see if your hypothesis is correct

7. The Scientific Method →The Social Scientific Method Test the predictions of the hypothesis using experiments  Find data to test the relationships between key variables • Experiments are hard to come by in the social sciences • Human subjects are messy • Setting up a controlled environment is difficult • Natural experiments? • Instead of experiments, the answer is data • Statistical methods can approximate experiments • Clear definitions of concepts are key

8. A Simple Example • Make an observation • “Twitter appears to have played an important role in the recent election, based upon anecdotal media reports and personal observations.” • Formulate a hypothesis • “Twitter increases flows of information and raises the informational awareness of voters about topics related to elections.” • Make a prediction based on the hypothesis • “The more that a news story is mentioned on Twitter, the more aware that voters will be about that news story.” • Find appropriate data to test this prediction • What data would we need to investigate this claim?

9. A Simple Example: Observation • Make an observation • “Twitter appears to have played an important role in the recent election, based upon anecdotal media reports and personal observations.” • This observation is about the role that Twitter plays in spreading information • We think it is important because we know that information plays an important part in the decision-making process for individual voters • Since Twitters appears to have an impact on information (based on our observation), it is worth exploring whether there is a measurable impact • In other words, Twitter seems to be important; the research question will be designed to find out if it actually is important

10. A Simple Example: Hypothesis • Formulate a hypothesis • “Twitter increases flows of information and raises the informational awareness of voters about topics related to elections.” • The hypothesis relates information on Twitter to the informational awareness/knowledge of voters in elections • The causal mechanism is left unstated, but could be a variety of things: • People are directly influenced by following Twitter feeds and reading tweets • People are indirectly influenced as other information sources they have (e.g., news reports, radio broadcasts, magazine stories) are influenced by stories being promulgated on Twitter • What other causal mechanisms might link information on Twitter to the knowledge of voters?

11. A Simple Example: Prediction • Make a prediction based on the hypothesis • “The more that a news story is mentioned on Twitter, the more aware that voters will be about that news story.” • This prediction relates a measure of information contained on Twitter to a measure of information held by voters • The main observable implication of this hypothesis is that more mentions of a news story on Twitter will lead to increased awareness of this story among voters • What other possible observable implications can we derive? • Could other media forms also show an increase of coverage about a particular story? • Do political campaigns respond to stories on Twitter with press releases, conference calls, or other information-related responses?

12. A Simple Example: Testing • Find appropriate data to test this prediction • Remember, the main prediction is that more mentions of a story on Twitter will lead to more knowledge of that story among voters • What data would we need to investigate this claim? • The claim is about the spread of information from Twitter to voters • We need to measure the information shared on Twitter • Counts of tweets? • We need to think about what content matter • What tweets should be included? • What time range? • All tweets everywhere? Just the U.S.? Just certain types of Twitter users? • Deciding how to measure information on Twitter is operationalizing this concept • We need to measure the information about which voters are aware • Surveys? • Other media? • How do we measure what people know? • Deciding what data are needed and finding that data are the hardest parts of research!

13. General Points on Research Design • Mixed methods analysis is the “gold standard” • Combination of quantitative and qualitative data • Could also include formal models • Mathematical representations of decisions • Often referred to as “game theory,” though in reality game theory is a subset of formal modeling • Matching the research design to the hypothesis under investigation is critical • How questions are asked and answered • What counts as evidence?

14. Executing a Research Project • Determine what kind of data will be needed • Quantitative • Large-N • Measurable in a clear and consistent way • Qualitative • Case studies • Not easily quantifiable • The Holy Grail of Social Science Research: Turning Quantitative Data into Qualitative Measures

15. Final Points • The social scientific research method is designed to answer questions or solve puzzles empirically • Theory and observation are used to develop a hypothesis • Evidence (i.e., data) is used to test the validity of the hypothesis • The hypothesis links theories with expected evidence • Data analysis can take many different forms • Quantitative: typically takes the form of statistical analysis • Qualitative: typically takes the form of case studies • There is an important role for all types of research

16. Contact Info Ashley Jester Data Services Coordinator Digital Social Science Center Social Sciences Libraries ashley.jester@columbia.edu