1 / 30

Week 3 of course “Quality and Ethics in Social Science Research” Matthijs Kalmijn

Value-free Research in the Social Sciences: Principles and Controversies 2017-2018 Semester 1, block 1. Week 3 of course “Quality and Ethics in Social Science Research” Matthijs Kalmijn University of Amsterdam, Department of Sociology. Content. Principles Merton’s classic norms

lwithrow
Download Presentation

Week 3 of course “Quality and Ethics in Social Science Research” Matthijs Kalmijn

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Value-free Research in the Social Sciences: Principles and Controversies2017-2018Semester 1, block 1 Week 3 of course “Quality and Ethics in Social Science Research” Matthijs Kalmijn University of Amsterdam, Department of Sociology

  2. Content • Principles • Merton’s classic norms • Objectivity • Bias (MacCoun) • Subgroup work & presentations • Controversies • Same-sex parenting: bad for kids? • Racial diversity: good for companies? • Remedies

  3. My own approach • Sociology (family, life course, inequality) • Quantitative • Survey based • Statistically analyze data to • Describe reality • Test theories about reality • Primacy of science

  4. The norms of scienceRobert K. Merton • Universalism • Communalism • Disinterestedness • Organized skepticism

  5. Examples of violations • Bourdieu said it …. • Epidemiological surveys in the Netherlands • Undocumented research data • Theoretical paradigms

  6. Objectivity in research • Objective / subjective is a continuum • Don’t throw out the baby with the bathwater • Aim for more rather than less objectivity • Distinguish • Subjectivity in the research questions we choose • Subjectivity in the theories we wish to test • Subjectivity in the use of research findings • Distinguish • Subjectivity of the researcher • Subjectivity of the things/people we study

  7. Value free science • Political and moral values may motivate you to do research • Political and moral values may distort your interpretation of research results • The problem of value free science is part of a more general problem of biased evidence interpretation

  8. Biased evidence interpretation: MacCoun’s typology

  9. Example 1 • Mental contamination

  10. Example 2

  11. Disorder & prejudice WARNING FRAUDULENT! Stapel, D. A. and S. Lindenberg. 2011. "Coping with Chaos: How Disordered Contexts Promote Stereotyping and Discrimination." Science 332(6026):251-53.

  12. DiederikStapel’smethod • PI works out theory and hypotheses in detail with collaborator • Collaborator develops full questionnaires for school-based survey experiment (makes several revisions) • Printing of questionnaires, purchase of incentives and other research material • School-based field stage (PI is only contact) • Questionnaires put in Excel by school pupils • Data back via PI to collaborator, starts analyzing • Hypotheses confirmed (“you have gold there”) • Elaborate co-writing stage and submission (e.g., JPSP)

  13. DiederikStapel’smethod • PI works out theory and hypotheses in detail with collaborator • Collaborator develops full questionnaires for school-based survey experiment (makes several revisions) • Printing of questionnaires, purchase of incentives and other research material • School-based field stage (PI is only contact) • Questionnaires put in Excel by school pupils • Data back via PI to collaborator, starts analyzing • Hypotheses confirmed (“you have gold there”) • Elaborate co-writing stage and submission (e.g., JPSP)

  14. Other examples of fraud • Biological science (Hwang Woo-Suk, stem cell research) • Medical research (Potti, cancer research) • Anthropology (Mart Bax, wars in Bosnia) • Political science (LaCour & Green, on how contact changes homophobia)

  15. Example 3 • Statistical significance • Is b significant? • b has a t-value which has a p-value, e.g. 4% • Meaning: Suppose the real b (effect) is 0, the chance of finding that value of b (or higher) is 4% • Since this chance is low (below 5%), we reject the null hypothesis (0 effect unlikely) • We call the observed b significant

  16. Disconfirmation bias Masicampo, E. J. and D. R. Lalande. 2012. "A Peculiar Prevalence of P Values Just Below .05." Quarterly Journal of Experimental Psychology 65(11):2271-79.

  17. Summary

  18. Motivation for bias • Self-interest • Interests of (external) organization • Political and moral views • Societal effects/consequences • Overcommitted to a theory

  19. Subgroup work

  20. Subgroup work • Subgroups formed based on discipline and/or substantive research interests • Work out examples of biases that can arise in your own research, at least one for each cell • These examples can be fictitious but must be ‘realistic’ (how could it go wrong….) • Make a Powerpoint and present back to class (one presenter) • Send to matthijskalmijn@gmail.com

  21. Controversies 1: Regnerus (2012) Criticized by Amato, Eggebeen and Osborne and others

  22. Regenerus ‘motivated’ or not? "As Christians, our lives should reflect our relationship with God and our desire to glorify Him," Regnerus says. "I've noticed that some Christian professors see a disconnect between their faith and their profession. I believe that if your faith matters, it should inform what you teach and what you research.” (Old quote in Trinity Christian Vollege Alumni Magazine)

  23. Replication by Cheng and Powell 2015

  24. Controversies 2: Herring Criticized by Stojmenovska, Bol & Leopold

  25. Herring ‘motivated’ or not? “Yes, diversity is still a good thing, but not just because it is related to business outcomes. Diversity is also a good thing because it reinforces the belief that everyone—no matter their race, ethnicity, gender, sexuality, or religion—deserves an equal opportunity. This message is even more important now than it was two presidential elections ago.” (in Herring’s response to Stojmenovska, Bol & Leopold)

  26. Replication by Stojmenovska et al.

  27. RACIAL DIVERSITY COMPANY SIZE CONFOUNDING VARIABLE SPURIOUS! COMPANY SALES

  28. Some remedies – institutional • Socialize students • Take an oath (?) • Let social scientists compete • Organize peer review • Share data • Conduct meta analyses

  29. Some remedies – individual • Engage alternative hypotheses • Strive for accuracy and precision • Look (also) for falsification options • Allow (and do) replication studies • Have respect for the annoying facts

  30. Moral and political values are sometimes good motivators but always poor guides

More Related