1 / 24

Sample Design and Issues with Telephone and Multi-Mode Surveys

Sample Design and Issues with Telephone and Multi-Mode Surveys. Meena Khare National Center for Health Statistics, COCHIS NCHS/DUC , Washington, D.C. July 12, 2006

paul-rich
Download Presentation

Sample Design and Issues with Telephone and Multi-Mode Surveys

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Sample Design and Issues with Telephone and Multi-Mode Surveys Meena Khare National Center for Health Statistics, COCHIS NCHS/DUC , Washington, D.C. July 12, 2006 “The findings and conclusions in this paper are those of the author(s) and do not necessarily represent the views of the National Center for Health Statistics, Centers for Disease Control and Prevention.”

  2. Outline • Telephone surveys • Sample design • Trends in response rates • Weighting and estimation • Issues with telephone and multi-mode surveys • Summary 3

  3. Telephone Surveys • Lower cost and rapid estimates • Sampling Frame: listed or unlisted telephone numbers with landline phone connections • Wireless numbers are excluded for legal reasons • Stratified by area codes and exchanges within selected geographic areas • Telephone numbers selected from the banks of 100 consecutive numbers within an exchange • Multiple call attempts made to resolve numbers as residential, business, nonworking, ring-no-answer, etc. 4

  4. Sample Design • Single stage cluster design • Sampling frame: Replicates of 100 banks [(xxx)-xxx-xx00 – (xxx)-xxx-xx99] with at least one residential number (1+Banks) • List assisted Random-Digit-Dialing (matching with ‘white pages’) • Independent random samples within geographic areas • Simultaneous screening and interviewing of households with landline telephone • Advance letters (and may be prepaid incentives) to telephone numbers with valid residential address (by reverse matching) • Multi-mode surveys: RDD surveys followed by a mail, Web or FTF survey of respondents and/or nonrespondents 5

  5. Distribution of Adults With and Without Landline Telephones Inside House and Wireless Telephone Service by Age, 2004 NHIS 6

  6. Nontelephone Households: Wireless-Only and Phoneless Households • Wireless only households (6-10%) • Men • Younger adults • Renting home • Uunrelated roommates • Income at or above poverty • Non-minority • Phoneless households (~2-5%) • Men • Older adults • Less educated • Lower income • Unemployed • Uninsured • Minority • References: Blumberg et al. (AJPH, 2006) 7

  7. Survey Operations (an example) Replicate of landline telephones Numbers with business, nonworking and other non-residential numbers removed 8

  8. Guidelines for Computing Response Rates Standards for computing response rates from AAPOR "Standard Definitions: Final Disposition of Case Codes and Outcome Rates for Surveys" , 4th edition, 2006. Definition of CASRO* Response Rates “On the Definition of Response Rates”, A Special Report of the CASRO Task Force on Completion Rates, 1982. Ezzati-Rice, T.M., Frankel, M.R., Hoaglin, D.C., Loft, J.D., Coronado, V.G., and Wright, R.A. “An alternative measure of response rate in random-digit-dialing surveys that screen for eligible subpopulations” Journal of Economic and Social Measurement, 26, 99-109, 2000. *CASRO: Council of American Survey Research Organizations 9

  9. RDD Response Rate CASRO* Response Rate = Resolution Rate x Screener completion Rate x Household Interview Completion Rate where * CASRO: Council of American Survey Research Organizations 10

  10. Trends in CASRO Response Rates from Selected RDD Surveys * A 1998 study of 35 RDD surveys reported: CASRO response rates ranged 22.2% - 70.1%, O’Rourke and Johnson, 1999 11

  11. NIS 2004 Sample • 3,607,627 phone numbers selected • 3,023,174 (83.8%) phone numbers resolved as residential • 32,638 households screened in with eligible children aged 19-35 months • 30,019 (92.0%) of eligible households with completed interviews • 73.1% CASRO response rate • 30,987 age-eligible children with completed interviews (87.1% with consent to contact provider) • 21,998 (71.0%) children with completed interviews and ‘adequate’ provider data (including unvaccinated children) 12

  12. Trends in National Immunization Survey Response Rates , 1995-2004 NIS 13

  13. Trends in Refusal and Partial Interview Cases, 1995-2004 NIS 14

  14. Trends in Rates of Refusal Conversion Among Children with Completed Interviews, 1995-2002 NIS 15

  15. Trends in Percent of Households Mailed an Advance Letter, NIS 1996-2004 Range by 78 sampling areas, 2002 NIS : 31.4% - 72.0% 16

  16. Sample Weights • RDD level interview weights are adjusted for • Probability of selection of telephone numbers • Unresolved numbers (using exchange information) • Nonresponse to screener and household interview • Multiple landline phone lines per household • Noncoverage of households with no landline telephone • Keeter’s Method (using information on interruptions in telephone service) • Noncoverage of population subgroups (post-stratification and raking) 17

  17. Estimation • Sampling weights should be used for computing estimates • For variance and interval estimates: geographic areas are used as Strata and sampled persons or households within an area are used as PSUs. • Available software for analyzing data from complex surveys • SUDAAN • SAS • STATA • SPSS (Complex Sample Module) • WESVAR 18

  18. Issues Related to Telephone Surveys • Noncoverage of nontelephone households • Changing telephone environment (e.g.,VOIP) • Wireless phone penetration • Undercoverage of households with landline telephone • Call-screening and call-blocking devices • Number portability: • mixed use of landline and wireless numbers • mixed-use telephone exchanges within a geographic area • Unknown residential status • Decline in Resolution rate 19

  19. Issues Related to RDD Surveys (cont…) • Multiple landline telephones in a household; mixed usage • Government initiatives (legal issues) • FCC: Do-not-call registry • Advance letters: missing for >40%; no address for unlisted numbers • Increase in refusals to participate and partial interviews • ERB mandate of only one additional call to refusals Decline in overall survey response rate 20

  20. Issues with Multi-mode Surveys • Incomplete sampling frames • Noncoverage and under-coverage of population sub-domains • Legal and financial issues • Increase in noncontact • Nonresponse adjustments and combining estimates • Availability of population controls • Potential for bias in estimates 21

  21. Summary • Telephone surveys are the most effective ways to obtain rapid estimates • Most RDD surveys are experiencing decline in response rates and decline is primarily due to decline in resolution rates • Increase in refusal rates and partial interview cases • Refusal conversion, advance letters, and incentives have helped in improving response rates 22

  22. Summary (cont…) • Use sampling weights for computing population estimates • To compensate for clustering, noncoverage and nonresponse • Refer to data user’s guide and other documents before analyzing survey data • Use software for analyzing complex sample data for estimating variance 23

  23. Thank You Contact:Meena KhareEmail:mkhare@cdc.gov 24

More Related