Sampling Procedures

1. Sampling Procedures January 24 & 26, 2011

2. Objectives By the end of this meeting, participants should be able to: • Distinguish the techniques behind and advantages of various kinds of sampling procedures. • Evaluate a survey sample based on common problems that should be avoided. • Calculate and interpret the margin of error for survey findings.

3. Sampling Suppose that you wanted to know how the American people felt about the war in Afghanistan. What would you do? • Obviously asking the entire public the question would be prohibitively expensive. • Fortunately, you would not have to. A carefully created and implemented survey of a few thousand people would give you a good reflection of the views of the entire country.

4. Sampling Method • First thing a researcher needs to do is decide what is the population of interest. • From there a researcher needs to decide what is the sampling frame.

5. Types of Sample There are two broad types of sample • Non-probability sample • Probability sample

6. Types of Non-probability Samples • Typical sample- choosing people that seem typical based on various demographic factors • Purposive sample- choosing people deliberatively possibly based on the advice of others • Volunteer subjects- surveying people that volunteer (American Idol, various insta-polls)

7. Types of Non-probability Samples • Haphazard sampling- choosing those people that are easiest to contact (Literary Digest 1936 poll, class polls, street corner polls) • Quota Sampling- choosing people that perfectly reflect the population of study (minorities, women, homosexuals, etc. ) • Snowball sampling- interviewing a random sample and asking them to identify people that they know that fit certain criteria. Generally used for rare populations

8. Probability Sampling • Despite their widespread usage non-probability samples are almost universally inferior to probability samples due to the introduction of bias • Probability sampling means that each person in the population has a known probability of being chosen (although not necessarily equal)

9. Types of Probability Samples • Simple random sample- using a list of the entire population, a random group is chosen • Systematic selection procedure- using a list of the entire population, a random number is selected and that many units are skipped between interviews • Need to avoid periodicity • Hard to generate lists

10. Types of Probability Samples • Stratified sample- divide the population in pieces and randomly sample within those pieces (regions, counties, dorms, etc.; this is the method used for election exit polls) • Cluster sample- divide the population into clusters (think neighborhoods) and conduct several interviews in each cluster. This introduces bias but can significantly reduce cost.

11. Types of Probability Samples • Multistage area sample- a portion of the geographic area is sampled, followed by a sampling of areas within the selected areas. The areas will be weighted based on their population. • Hybrid sampling- a combination of any of the previous sampling methods. Such types include Multiple Frame Designs, repeated attempts to sample the same population and Parallel Samples, comparing a baseline sample to another sample.

12. Telephone Samples • Telephone surveys are one of the most common types of surveys • They can be potentially difficult because not all people are listed and telephone books are frequently out of date. • Some researchers use a method called add a digit dialing. In this method, numbers are chosen at random from the directory and a digit is added to that number

13. Telephone Samples • Another method is random digit dialing. A computer will generate a telephone number at random (perhaps an area code, prefix or just a suffix). This method needs roughly 5 numbers for each number in the sample due to the high rate of failure • Telephone samples suffer from multiple phone lines in each home, refusal, difficulty reaching any one, etc. • Bias? • Alternative methods?

14. Telephone Samples • One issue that faces telephone surveys is which person to speak to within the household. One of the most common methods used in the next birthday method.

15. Problems in Sampling Generally random sampling will give a sample that reflects the broader population. There are still potential problems that need to be considered. • Noncoverage error- parts of the population may not be covered in the sample (for example, people without phones, the infirmed, etc. ) • Sampling the wrong population- the precise population needs to be sampled not just a part of it

16. Problems in Sampling • Response rate- lower response rates are not necessarily a problem unless those that refuse have similar demographic factors. • Sampling error- this is the error inherent in using a sample to generalize to a broader population

17. Sampling Error We need to consider sampling error: • The error that arises from trying to represent a population with a sample. • Sampling error does not include other sorts of error that can result from surveys. We often speak of a 95% confidence interval. • If repeated samples were taken, 95% of the samples would contain results within the margin of error. • “A statistician would say that we are taking a 5% chance of drawing a faulty conclusion…” (WKB, 68).

18. Margin of Error Margin of error (forumula on board) a) Where, t =1.96 for large samples b) The value 1.96 comes from our understanding of the distribution of possible values. c) f is the sampling fraction (or the fraction of the population that is being sampled), (1-f)1/2 is ignored a) when sampling with replacement b) or when the population is very large

19. Margin of Error d) p is the sample proportion (for example, the proportion approving of President Obama). • p and (1-p) may be written in either proportion (0 to 1) or percentage (0 to 100) terms • Decide which based on what unit you want the margin of error to be expressed in.

20. Margin of Error: Example • Suppose there are 1,872 political science majors on campus. We randomly select 250 and ask them whether they watched TV last night. 66 percent of the sample respond “yes.” • How confident can we be that this percentage reflects our underlying population of interest (political science majors)? • Recall: margin of error formula • p = .66 and (1-p)= .34; for percentages: p=66 and (1-p)=34 • n = 250 and f = 250/1872 • Margin of error=±1.96[(66×34)/249]1/2×[1−(250/1872)]1/2= 5.5% • We are 95 percent confident that 66 ± 5.5% of social science majors watched TV last night. • Or, we can say, we are 95% confident that the true population percentage is in between 60.5% and 71.5%

21. For January 31 • Read WKB chapter 4. • Answer the following questions: • Imagine we conducted a survey of 350 UGA students and found that 93% thought the Bulldogs would win the SEC championship. What is the margin of error (at 95%)? Based on that margin of error, we can be 95% sure that championship predictions lie in what range? • Imagine we conducted a survey of GA voters and found that Obama had approval of 55%. 750 people were surveyed, what is the margin of error (at 95%)? Based on that margin of error, we can be 95% sure that Obama’s support in GA lies in what range?