Chapter 13 Samples and Surveys

1. Chapter 13Samples and Surveys

2. 13.1 Two Surprising Properties of Sampling • How is the winning car model of J.D. Power • and Associates Initial Quality Award • determined? • By focusing on a subset of the whole group (a sample) • By making sure that items are selected randomly from the larger group

3. 13.1 Two Surprising Properties of Sampling • Definitions • Population: the entire collection of interest • Sample: subset of the population • Survey: posing questions to a sample to learn about the population • Representative: samples that reflect the mix in the entire population • Bias: systematic error in selecting the sample

4. 13.1 Two Surprising Properties of Sampling • The two surprises are: • The best way to get a representative sample is to pick members of the population at random. • Larger populations do not require larger samples.

5. 13.1 Two Surprising Properties of Sampling • Randomization • A randomly selected sample is representative of the whole population. • Randomization ensures that on average a sample mimics the population.

6. 13.1 Two Surprising Properties of Sampling • Comparison of Two Random Samples from a • Population of 3.5 Million Customers.

7. 13.1 Two Surprising Properties of Sampling • Randomization • Produces samples whose averages resemble those in the population (avoids bias). • Enables us to infer characteristics of the population from a sample.

8. 13.1 Two Surprising Properties of Sampling • Infamous Case: The Literary Digest • The Literary Digest predicted defeat for Franklin D. • Roosevelt in the 1936 presidential election. They • selected their sample from a list of telephone • numbers (telephones were a luxury during the • Great Depression). Roosevelt’s supporters tended • to be poor and were underrepresented in the • sample.

9. 13.1 Two Surprising Properties of Sampling • Sample Size • Common sense tells us that bigger is better. • Surprisingly, the size of the population (unless it is small) does not influence the sample size to be used.

10. 13.1 Two Surprising Properties of Sampling • Simple Random Sample (SRS) • A sample of n items chosen by a method that has an equal chance of picking any sample of size n from the population. • Is the standard to which all other sampling methods are compared.

11. 13.1 Two Surprising Properties of Sampling • Simple Random Sample (SRS) • Sampling Frame: a list of items from which to select a random sample. • Systematic Sampling: method for selecting items from a sampling frame that follows a regular pattern (e.g., every 10th item).

12. 13.1 Two Surprising Properties of Sampling • Identifying the Sampling Frame • If there is no fixed population of outcomes, no sampling frame exists (e.g., output from a production process). • The list available may differ from the list desired (e.g., voter registration lists identify people who can vote, not those who will).

13. 13.2 Variation • Estimating Parameters • Parameter: a characteristic of the population (e.g., µ) • Statistic: an observed characteristic of a sample (e.g., ) • Estimate: using a statistic to approximate a parameter

14. 13.2 Variation • Notation for Statistics and Parameters

15. 13.2 Variation • Sampling Variation • Is the variability in the value of a statistic from sample to sample. • The price we pay for working with a sample rather than the population.

16. 13.2 Variation • Sampling Variation in Sample Means

17. 13.2 Variation • Sampling Variation in Survey Results • Change Over Time

18. 4M Example 13.1: EXIT SURVEYS • Motivation • Why do customers leave a busy clothing • store in the mall without making a purchase?

19. 4M Example 13.1: EXIT SURVEYS • Method • A survey is necessary. The owner decides to • survey 50 weekend customers. The ideal • sampling frame would list every customer • who did not make a purchase over the • weekend. Such a list does not exist.

20. 4M Example 13.1: EXIT SURVEYS • Mechanics • Interview every 20th departing shopper on • both Saturday and Sunday. Based on typical • customer flow, a sample of size 60 is • expected. Ask customers why they didn’t • make a purchase. Keep a record of • nonresponses.

21. 4M Example 13.1: EXIT SURVEYS • Message • On the basis of the survey, the owner will be • able to find out why shoppers are leaving • without buying.

22. 13.3 Alternative Sampling Methods • Stratified Samples • Divide the sampling frame into homogeneous groups, called strata • Use simple random sample to select items from each strata

23. 13.3 Alternative Sampling Methods • Cluster Samples • Divide a geographic region into clusters • Randomly select clusters • Randomly choose items within selected clusters

24. 4M Example 13.2: ESTIMATING THE RISE OF PRICES • Motivation • What goes into determining the consumer • price index (CPI), the official measure of • inflation?

25. 4M Example 13.2: ESTIMATING THE RISE OF PRICES • Method • The Bureau of Labor Statistics (BLS) uses a • survey to estimate inflation. The target • population consists of the costs of every • consumer transaction in urban areas during a • specific month.

26. 4M Example 13.2: ESTIMATING THE RISE OF PRICES • Mechanics • The BLS has a list of urban areas and a list • of people living in each, but does not have a • list of every sales transaction. So the BLS • divides items sold into 211 categories and • estimates the change in price for each • category in every area.

27. 4M Example 13.2: ESTIMATING THE RISE OF PRICES • Message • The urban consumer price index is an • estimate of inflation base on a complex, • clustered sample in selected metropolitan • areas.

28. 13.3 Alternative Sampling Methods • Census • A comprehensive survey of the entire population. • Cost and time constraints generally prohibit carrying out a census; in some cases a census is not feasible.

29. 13.3 Alternative Sampling Methods • Voluntary Response • A sample consisting of individuals who volunteer when given the opportunity to participate in a survey. • These samples are biased toward those with strong opinions.

30. 13.3 Alternative Sampling Methods • Convenience Samples • A sampling method that selects individuals who are readily available. • Although easy to obtain, these samples are rarely representative.

31. 13.4 Checklist for Surveys • Questions to Consider • What was the sampling frame? • Is the sample a simple random sample? • What is the rate of nonresponse? • How was the question worded? • Did the interviewer affect the results? • Does survivor bias affect the survey?

32. 13.4 Checklist for Surveys • Differences in Survey Results about Health Care Reform in 2010 Due to Question Wording

33. Best Practices • Randomize. • Plan carefully. • Match the sampling frame to the target population.

34. Best Practices (Continued) Keep focused. Reduce the amount of nonresponse. Pretest your survey.

35. Pitfalls • Don’t conceal flaws in your sample. • Do not lead the witness. • Do not confuse a sample statistic for the population parameter. • Do not accept results because they agree with what you expect.