Data Collection: Enhancing Response Rates while Limiting Errors

1. Data Collection: Enhancing Response Rates while Limiting Errors Chapter 10, Student Edition MR/Brown & Suter

2. Learning Objectives • Describe the five types of error that can enter a study • Give the general definition for response rate • Discuss several ways in which response rates might be improved MR/Brown & Suter

3. Learning Objectives • Describe the five types of error that can enter a study • Give the general definition for response rate • Discuss several ways in which response rates might be improved MR/Brown & Suter

4. Learning Objective 1 Sampling Error Noncoverage Error Nonresponse Error Response Error Office Error MR/Brown & Suter

5. Learning Objective 1 • Sampling error – the difference between results obtained from a sample and results that would have been obtained had information been gathered from or about every member of the population • Sampling error is decreased by increasing sample size • Can be estimated (assuming probability sample) • Usually less troublesome than other kinds of error MR/Brown & Suter

6. Learning Objective 1 • Noncoverage error – nonsampling error that arises because of a failure to include some units, or entire sections, of the defined target population in the sampling frame • Noncoverage error is basically a sampling frame problem • Can be reduced, although not necessarily eliminated, by recognizing its existence and working to improve the sampling frame MR/Brown & Suter

7. Learning Objective 1 • Nonresponseerror – nonsamplingerror that represents a failure to obtain information from some elements of the population that were selected and designated for the sample • This is a potential problem that only occurs when those who do respond are systematically different in some important way from those who don’t respond • Example – A university wants to assess the success of its graduates, based on their annual salaries, five years after graduation • Which graduates are more likely (less likely) to return their survey? Those who are happy (unhappy) with their salaries. MR/Brown & Suter

8. Learning Objective 1 • Response error occurs when an individual provides a response to an item, but the response is inaccurate for some reason • Possible causes of response error include • Does the respondent understand the question? • Does the respondent know the answer to the question? • Is the respondent willing to provide the true answer to the question? • Is the wording of the question or the situation in which it is asked likely to bias the response? MR/Brown & Suter

9. Learning Objective 1 • Office error – nonsampling errors that arise in the editing, coding, or analysis phases of research • Most office errors can be reduced, if not eliminated, by exercising proper controls in data processing MR/Brown & Suter

10. Learning Objectives • Describe the five types of error that can enter a study • Give the general definition for response rate • Discuss several ways in which response rates might be improved MR/Brown & Suter

11. Learning Objective 2 • Response rate is defined as the number of completed interviews with responding units divided by the number of eligible responding units in the sample • The general response rate calculation is • RR = CI/E • RR = Response Rate; • CI = Number of Completed Interviews with Responding Units; • E = Number of Eligible Responding Units in the Sample MR/Brown & Suter

12. Learning Objective 2 • The response rate calculation for Web-based and mail surveys is • RR = UQR/(CA - BA) • RR = Response Rate; • UQR = Number of Usable Questionnaires Returned; • CA = Number of Contacts Attempted; • BA = Number of Bad Addresses MR/Brown & Suter

13. Learning Objective 2 • The response rate calculation for telephone surveys (no eligibility requirement) is • RR = CI/(CI + R + NAH) • RR = Response Rate; • CI = Number of Completed Interviews; • R = Number of Refusals; • NAH = Number of Not-At-Homes MR/Brown & Suter

14. Learning Objective 2 • The response rate calculation for telephone surveys (with eligibility requirement) is • RR = CI/(CI + E% (R + NAH)) • RR = Response Rate; • CI = Number of Completed Interviews; • E = Percentage of Eligible Interviewees • R = Number of Refusals; • NAH = Number of Not-At-Homes • E% = CI/(CI + IE) • IE = Number of Ineligible Interviewees MR/Brown & Suter

15. Learning Objectives • Describe the five types of error that can enter a study • Give the general definition for response rate • Discuss several ways in which response rates might be improved MR/Brown & Suter

16. Learning Objective 3 • Survey Length • Guarantee of Confidentiality or Anonymity • Interviewer Characteristics and Training • Personalization • Response Incentives • Follow-Up Surveys MR/Brown & Suter