1 / 33

Introduction to Research Design

Introduction to Research Design. StatLab Workshop Fall 2011 Brian Fried and Jeremy Green. Outline of Workshop. Part I: Causation and Statistics What is Causation? Correlation? Why Statistics? Threats to Inference Part II: Gathering and Using Data Gathering Data Managing Data

cameo
Download Presentation

Introduction to Research Design

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Introduction to Research Design StatLab Workshop Fall 2011 Brian Friedand Jeremy Green

  2. Outline of Workshop Part I: Causation and Statistics • What is Causation? Correlation? • Why Statistics? • Threats to Inference Part II: Gathering and Using Data • Gathering Data • Managing Data Part III: Writing with Statistics • A General Outline, with an example

  3. Causation vs. Correlation Causation… …correlation

  4. Why Statistics • Probabilistic Relationships(see previous graph) • Multivariate RelationshipsWe can analyze the relationships between multiple variables at the same time.(e.g. education, age, gender, income …. -> voting) • What is a regression?

  5. When not to use statistics • Insufficient observations. • Observations are not easily comparable. • …. Other methods may still be appropriate (historical, interpretative, qualitative, constructivist to name a few in the social sciences).

  6. Threats to Inference • Endogeneity (vsexogeneity of errors) • Autocorrelation (time series) • Homo/Heteroskedasticity • Internal vs. external validity Probably the most important step in research design; advanced techniques can often compensate.

  7. Part II: Data Think about analyses early! (Ideal vs. Possible) What’s Possible? What’s Convincing? • Experimental Ideal • Practical Data Limitations • Collecting Your Own Data • Using Other Data Some data sources: • StatlabWebpage (http://statlab.stat.yale.edu) • Advisors/Professional Contacts • Yale StatCat (http://ssrs.yale.edu/statcat/) • ICPSR (http://www.icpsr.umich.edu) • Reference Librarian (Julie Linden) • Public sources (generally government or academic)

  8. Data Types and Uses Dependent Variable (response, outcome, criterion) Independent Variables (explanatory or predictor variables) Control / Confounding Variables Categorical and Continuous Variables Individual vs. Aggregate Data Remember: Types of variables we choose determine the statistics we use Qualitative knowledge always helps!

  9. Once You’ve Found or Collected Your Data Download the data and documentation • StatTransfer (Statlab) Determine data file type • Often a text file (.txt, .dat, .raw) • Other formats (.csv, .sas, .xls) Converting text & delimited files Choose a statistical software program • SPSS a good place for basic analysis

  10. Managing your data Back up all Master Data Files Codebook • Merging Data • Adding variables, cases, computing new variables Keep a roadmap • Keep a log of all analyses with what you have done • Save syntax files

  11. Syntax Files What are they? Text-files used to enter commands in bulk Why? You will make mistakes, need to make changes Many statistical programs let you use pull down menus How do I know what to write? Program manuals (help screens, online documentation, hard copy books) provide the underlying commands

  12. Part III: Writing Introduction Theory (Lit Review) Data Description Analysis/Results Conclusion

  13. Introduction Question What is the question you want to answer? Why should we care? Audience Hypothesis Succinctly state your claim Context & Summary

  14. Motivation An Illustrative Example: BolsaFamilia • Broad Question: Are politics becoming more programmatic in Brazil? • Testable hypothesis/question: Is BolsaFamilia, a conditional cash transfer (CCT) program that benefits a quarter of Brazil’s population, programmatic?

  15. An Illustrative Example: BolsaFamilia Programa Bolsa Família – key facts • Conditional cash transfer (CCT) program, launched in October 2003. This was not the first CCT program in Brazil; some existing programs (like BolsaEscola) were incorporated into BolsaFamilia. • Benefits families with per capita income below US$78. • 12 million poor families (almost 50 million people) currently receive support in all 5,564 Brazilian municipalities; • Size of stipend: between US$13 and US$114, depending on the family’s size and poverty level. • Average amount: US$54 per family • 2009 Budget: US$ 10.5 billion (0.4% of Brazil’s GDP)

  16. Theory/Lit. Review What does existing theory say? • What do you believe? • Position yourself within theoretical debates. Identify Testable Hypotheses Choose Method Best Suited to Testing Your Hypothesis Do you need statistics after all? • Quantitative vs. Qualitative research

  17. Research Question An Illustrative Example: BolsaFamilia Do political criteria explain the variation in BolsaFamilia’s coverage across municipalities? • Theoretical (Cox and McCubbins 1986, Dixit and Londregan 1996, Lindbeck and Weibell 1987) and empirical (Ames 1987, Levitt and Snyder 1995, Schady 2000, Dahberg and Johansson 2002, Stokes 2004, Kitschelt 2010) reasons to believe that political spending is often targeted, especially given Brazil’s history with clientelism and pork.

  18. How do politicians target? An Illustrative Example: BolsaFamilia • “Core” • “Swing” • Mobilization

  19. Descriptive Statistics Variables Dependent Variable(s) Explanatory (Independent) Variable(s) Control (Independent) Variable(s) Graphs Summary Statistics on Key Variables Number, Mean, Minimum, Maximum, Standard Deviation Cross-Tabs

  20. Descriptive Statistics An Illustrative Example: BolsaFamilia

  21. DescriptiveFigures An Illustrative Example: BolsaFamilia

  22. Descriptive Statistics An Illustrative Example: BolsaFamilia

  23. An Illustrative Example: BolsaFamilia Key Variables

  24. So, how do I analyze my data? Correlational design • Correlation allows you to quantify relationships between variables (r, r-squared) • Correlation, partial correlation • Regression allows you predict scores on 1 variable from subjects score on another variable(s) Group differences • t-test & ANOVA • Chi-square for categorical and frequency data Significance v. effect size Simulations

  25. Methods of Analysis(Empirical Strategy) We discussed this in Part I, but one generally devotes a section to explaining how one will identify a causal relationship prior to the results section. Coverage = β0 + β1(political criteria) + βXX + e

  26. Results: Explaining Coverage in 2009 An Illustrative Example: BolsaFamilia

  27. Effect of Standard Deviation Shift of Explanatory Variables on Coverage in 2009 An Illustrative Example: BolsaFamilia

  28. Robustness Identify Threats to Inference! (Do I have any?)

  29. Robustness Check: Relationship between Coverage in 2004 and Prior Elections An Illustrative Example: BolsaFamilia

  30. Putting Output into a Paper Cut and Paste Graphs Cut and Paste into Word Processing document Save as .jpeg or .tif file Tables Cut and Paste Format in Word Processing document Import into Excel, format, and then place in Word

  31. More Advanced Analysis Multivariate techniques are only a start; they do help to account for confounding factors, allow for testing change over time and more complex hypotheses… • Be honest about your abilities. • Ask for help • Best off including techniques that you fully understand, but may be worth learning something new!

  32. Take Away Messages • Begin by thinking about what question interests. • Look for data and consider appropriate methods; identify what hypotheses are actually testable. • Design and run analysis; keep a codebook/syntax files! • Back up data • Ask for help-especially when choosing method—and seek feedback on research design. • Research and Writing an Iterative Process

  33. Feedback Please give us feedback! (Go to the StatLab’s workshop page, click on “we welcome feedback”, and select “Introductory Research Design” Or go to:statlab.stat.yale.edu/workshops/feedback.jsp)

More Related