1 / 61

Presented at AAPOR 2008 New Orleans, LA May 16, 2008

Who Needs RDD? Combining Directory Listings with Cell Phone Exchanges for an Alternative Sampling Frame. Presented at AAPOR 2008 New Orleans, LA May 16, 2008. Thomas M. Guterbock TomG@virginia.edu James M. Ellis jme2ce@virginia.edu Abdoulaye Diop adiop@virginia.edu Kien Le

tavia
Download Presentation

Presented at AAPOR 2008 New Orleans, LA May 16, 2008

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Who Needs RDD? Combining Directory Listings with Cell Phone Exchanges for an Alternative Sampling Frame Presented at AAPOR 2008 New Orleans, LA May 16, 2008

  2. Thomas M. Guterbock TomG@virginia.edu James M. Ellis jme2ce@virginia.edu Abdoulaye Diop adiop@virginia.edu Kien Le tkl7b@virginia.edu John Lee Holmes jlh2r@virginia.edu CSR—University of Virginia www.virginia.edu/surveys . . . A unit of the Weldon Cooper Center for Public Service

  3. The Research Problem:RDD under threat Are there good alternatives?

  4. RDD under threat Random Digit Dialing involves a certain degree of inefficiency Costs of this extra effort justified by completeness of coverage (at least until recently) Recent trends raising these costs: decreasing density of working numbers, increasing rates of non-contact, and rising rates of refusal Advent of cellular phone only households diminishes completeness of its coverage 4

  5. A “New Norm?” • Dual-frame “RDD+Cell” has arisen in response to these challenges • traditional list-assisted RDD sample with RDD of working cellphone exchanges. • To screen or not to screen? • And, if not, how to weight? • But other dual frames may also be worth exploring . . .

  6. Proposed Alternative: EWP+Cell • EWP+Cell = • “Electronic White Pages” + Cell Phone RDD • Promises considerably greater efficiency and cost savings over RDD+Cell • especially for specific, small geographic regions • or areas not co-extensive with any set of telephone Area Codes. • EWP+Cell fails to cover: unlisted landline households that have no cell phone • We will examine: How big a problem is that?

  7. Data Source:2006 National Health Interview Survey permits estimations of the size of . . . • the non-covered segment • demographic characteristics • health characteristics • degree of coverage bias

  8. What did we find?A Preview • Surprisingly little coverage bias to be expected from EWP+Cell • Potential cost savings from EWP+Cell compared to RDD+Cell

  9. A brief review of the research Not much literature or research compares directory-listed samples with list-assisted, landline RDD samples Consequently, the degree and nature of the differences between listed and unlisted households is not established. 9

  10. Older studies (before 2002) Most found only slight differences in substantive results between EWP and RDD frame samples Some efficiency gains in smaller geographic areas Some differences in demographics noticed And in 2007, Zogby announced plans to rely on EWP over RDD phone samples, citing lack of substantive differences in results. 10

  11. Recent studies on EWP vs RDD: Substantial differences shown • Unlisted rates are higher for: • Blacks, Hispanics • Lower income • Renters • Single people • See: Guterbock, Diop • and Holian (2007)

  12. From 3 segments to 5

  13. The universe of U.S. telephone households, 2006

  14. RDD samples cover all landline households, listed or not RDD Cell-phone- only households are excluded

  15. Cell phone samples include some that are also in the RDD frame Landline- only households are excluded Cell phones

  16. RDD and Cell samples overlap,yield complete coverage RDD CELL + LANDLINE 52.0% CELL ONLY 16.6% LANDLINE ONLY 31.4% Cell phones All percentages are from 2006 NHIS data.

  17. We need also to consider: listedness • Some landlines are listed in the residential directory or Electronic White Pages [EWP] • LLL = Listed Landline • Some landline households are unlisted • ULL = unlisted landline • LLL and ULL may or may not also have a cell phone in the household • Cell phones are unlisted by definition • Result: five segments of the telephone universe . . .

  18. Five telephone segments 2 CELL + ULL 17.7% 3 ULL ONLY 14.2% 1 CELL ONLY 16.6% 4 CELL + LLL 34.3% 5 LLL ONLY 17.2% See table I All percentages are from 2006 NHIS data.

  19. Five segments differ, sometimes sharply

  20. Segments differ on key demographics See table II All percentages are from 2006 NHIS data.

  21. Segments differ on key health questions See table II All percentages are from 2006 NHIS data.

  22. Our analysis deals withthree sampling frames: • EWP • List-assisted (landline) RDD • Cell phone RDD

  23. We examine 4 sampling designs: • 2 Single frame designs: • EWP only • Landline RDD

  24. We examine 4 sampling designs • 2 Dual Frame designs: • EWP+Cell • RDD+Cell

  25. Three design contrasts RDD+Cell is the base for all comparisons • It includes the full universe of phone HH We will compute coverage bias for each contrast: • EWP vs. RDD+Cell • RDD vs. RDD+Cell • EWP+Cell vs. RDD+Cell

  26. Formula for coverage bias Ῡ= mean for full population ῩC = mean for covered cases ῩU = mean for cases not covered U = cases not covered N = all cases

  27. Contrast I: EWP vs. RDD+Cell telephone samples

  28. The universe of U.S. telephone households

  29. EWP sample excludes unlisted landline and cell-only 2 EXCLUDED CELL + ULL 17.7% ῩU 3 EXCLUDED ULL ONLY 14.2% 1 EXCLUDED CELL ONLY 16.6% 4 CELL + LLL 34.3% 5 LLL ONLY 17.2% U/N = .485 EWP All listed landline phones ῩC

  30. Coverage bias table:EWP vs. RDD+Cell See table IV All percentages are from 2006 NHIS data.

  31. Coverage bias table:EWP vs. RDD+Cell See table IV All percentages are from 2006 NHIS data.

  32. Contrast II: RDD vs. RDD+Cell only telephone samples

  33. RDD samples cover all landline households, listed or not RDD ῩC ῩU

  34. RDD fails to cover 16.6% RDD CELL + LANDLINE 52.0% CELL ONLY 16.6% LANDLINE ONLY 31.4% U/N = .166 Cell phones All percentages are from 2006 NHIS data.

  35. Coverage bias table:RDD vs. RDD+Cell See table V All percentages are from 2006 NHIS data.

  36. Coverage bias table:RDD vs. RDD+Cell See table V All percentages are from 2006 NHIS data.

  37. Contrast III: EWP+Cellvs. RDD+Cell telephone samples

  38. RDD+Cell covers all phone households RDD CELL + LANDLINE 52.0% CELL ONLY 16.6% LANDLINE ONLY 31.4% Cell phones

  39. EWP + Cell Sample Design EXCLUDES ULL- ONLY Cell EWP All listed landline phones

  40. EWP + Cell excludes ULL-only households EXCLUDED: 3 ULL ONLY 14.2% 2 CELL + ULL 17.7% U/N = .142 1 CELL ONLY 16.6% ῩU 4 CELL + LLL 34.3% 5 LLL ONLY 17.2% ῩC

  41. Coverage bias table: EWP+Cell vs. RDD+Cell See table VI All percentages are from 2006 NHIS data.

  42. Coverage bias table: EWP+Cell vs. RDD+Cell See table VI All percentages are from 2006 NHIS data.

  43. Summary of 3 contrasts

  44. 3 contrasts: 2006 estimates (ῩC ) See table VII

  45. 3 contrasts: 2006 raw bias (ῩC -Ῡ) See table VIII

  46. 3 contrasts: 2006 percent bias See table IX

  47. Changes in CoverageNHIS 2003 - 2006

  48. unlisted landline only cell phone only Changes in telephone status over time

  49. Changes in percent bias over time: RDD vs. RDD+Cell (NHIS data)

  50. Changes in percent bias over time: EWP+Cell vs. RDD+Cell (NHIS data)

More Related