Chapter 10 Principles of Survey Research

1. Chapter 10Principles of Survey Research

2. Surveys: Answering Diverse Questions • Surveys are a regular feature of our lives now, one of the most widely used forms of research. • The first broadly applied surveys focused on presidential elections in 1936. It led to the wrong predictions because of poor sampling methods.

3. Surveys: Answering Diverse Questions Census versus Sample • Census—Data collection that involves every member of the population of interest • Survey—Data collection that involves a subset of the population

4. Controversy: The U.S. Census • The U.S. Census is designed to count every person in the country. • No census has ever counted everybody because people can be hard to find and may refuse to participate, even though they are required to by law • Statistics can accurately estimate the population size, but there are always political squabbles involved in doing so, another case in which science and culture interact to determine policies.

5. Surveys: Answering Diverse Questions • Accuracy of Survey Results • Survey researchers have refined their techniques so that surveys are generally pretty accurate. • Accuracy depends on how well the samples are chosen and on sample size. • Survey accuracy can be high even with samples that are modest in size, like 1000 people to estimate 50 million voters in a national election

6. Accuracy of Survey Results Survey accuracy can be high even with samples that are modest in size, like 1000 people to estimate 50 million voters in a national election

7. Ethics in Survey Research • Survey research is bound by ethical considerations, like any research. • It is standard to provide anonymity and confidentiality to respondents. • Ironically, stressing anonymity and confidentiality may arouse suspicions in respondents

8. Ethics in Survey Research • You have rights if you are surveyed in scientific research according to the ethics code developed by the Council for Marketing and Opinion Research

9. Selecting Your Methodology You need to identify and select your sample. • Sampling frame • A subset of the population from which the sample is selected • The nature of your sample can affect your research

10. Selecting Your Methodology

11. Selecting Your Methodology You need to identify the means of collecting data • Telephone interviews—widely used because they are inexpensive and efficient • In-person surveys—good for in-depth information but very expensive • Mail surveys—very inexpensive but troubled with low return rates • Internet surveys—good for reaching a wide-ranging sample, but with unknown properties because it involves a new technology

12. Selecting Your Methodology • Question Types • Open-Ended Question–A question that respondents answer using their own words, unconstrained by choices provided by the researcher • Closed-Ended Question–A question that contains a set of answers from which the respondent chooses • Both types of questions can lead to valid data; closed-ended questions are more common because they are easier to score and analyze, but they do not give as rich a set of responses as open-ended questions.

13. Selecting Your Methodology • Question Content • Memory Questions • People may give wrong answers because they intentionally lie • People may give wrong answers because they misremember

14. Selecting Your Methodology

15. Selecting Your Methodology

16. Selecting Your Methodology • Question Content • Attitude Questions • Chronically Accessible Information—Feelings and memories that a person can retrieve at any point • Temporarily Accessible Information—Feelings and memories that a person can retrieve only when cued for information (e.g., by previous questions on a survey).

17. Selecting Your Methodology

18. Response Bias • Response bias—A tendency for a respondent to answer in predictable ways, independent of question content. • Examples • Always agreeing with a statement • Always responding with a response from the middle of a scale • Responding in ways designed to make the person look good

19. Response Bias Studying Sensitive Issues • People will respond to sensitive questions • Assure respondents of anonymity (although this may not always be necessary) • Generate rapport with the respondent

20. Response Bias • Social Desirability Bias –Tendency of respondents to answer in ways that generate a positive impression of themselves • Impression management –Respondents actively misrepresent themselves in order to look favorable • Self-deception positivity –Respondents try to provide generally honest answers, but they are overly optimistic about themselves

21. Response Bias

22. Response Bias • Social Desirability Bias • New research suggests that social desirability may not occur to the extent previously believed

23. Response Bias • Acquiescence—In survey research, the tendency to agree with the assertion of a question, regardless of the content • Some people are predisposed to agree with what they think the researcher believes, so they tend to say “yes” in response to questions • Some people believe the researcher has higher status, so they want to agree with the person, saying “yes” to many questions

24. Response Bias • Satisficing–Tendency of people to be satisfied with the first acceptable response to a question, even if it is not the best response. It is likely to occur with • High task difficulty (e.g., hard questions) • Lower respondent ability (e.g., the person cannot generate good answers) • Low respondent motivation (e.g., a person is participating for extra credit and really does not care) • Optimizing–Tendency of people to search for the best response to a question

25. Response Bias

26. Controversy: Adolescent Smoking • Research suggests that 4 million adolescents smoke in the U.S., but interpreting the research is not straightforward • The research was government sponsored and was conducted by professionals who knew how to carry out such research

27. Controversy: Adolescent Smoking Defining terms clearly is important in understanding your research • What is an adolescent? • Generally 12-17 years of age • A 12-year old and a 17-year old smoking mean very different things.

28. Controversy: Adolescent Smoking • What does it mean “to smoke?” • The researchers had to come up with an operational definition, so they called a person a smoker who had one puff in the last 30 days • One puff shared among friends once is very different from a pack of cigarettes a day • 72% of the “smokers” smoked very little, many almost never • The researchers collected a lot of detailed information that isn’t addressed here, but the question of definitions is still important.

29. Controversy: Adolescent Smoking • Few younger adolescents smoked • About a quarter of older adolescents who smoked were “heavy smokers” (10 or more cigarettes a day for the past 20 days) • 31% of “smokers” had less than one cigarette when they did smoke, and they didn’t smoke often

30. Controversy: Adolescent Smoking Understanding research results • Definitions are important because they define the nature of the results • Complex problems cannot be reduced easily to simple numbers (e.g., 4 million smoking adolescents) • You have to put the numbers into appropriate context in order to understand them.

31. Sampling Issues • Adequate samples are necessary for valid research • Self-selected samples • Samples in which people actively seek out the chance to participate in research • This type of sample is considered biased, so results are not likely to generalize to the population

32. Sampling Issues • Hidden Populations—Populations that are hard to study because people may be engaged in secretive activities, so they do not want to be recognized as members of that population. • One useful approach to finding hidden populations is through chain-referral methods. • Set of sampling techniques in which people who know about a population provide access to that population

33. Chain-Referral Methods