Who Needs RDD?
This presentation is the property of its rightful owner.
Sponsored Links
1 / 61

Presented at AAPOR 2008 New Orleans, LA May 16, 2008 PowerPoint PPT Presentation


  • 57 Views
  • Uploaded on
  • Presentation posted in: General

Who Needs RDD? Combining Directory Listings with Cell Phone Exchanges for an Alternative Sampling Frame. Presented at AAPOR 2008 New Orleans, LA May 16, 2008. Thomas M. Guterbock [email protected] James M. Ellis [email protected] Abdoulaye Diop [email protected] Kien Le

Download Presentation

Presented at AAPOR 2008 New Orleans, LA May 16, 2008

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Presented at aapor 2008 new orleans la may 16 2008

Who Needs RDD? Combining Directory Listings with Cell Phone Exchanges for an Alternative Sampling Frame

Presented at

AAPOR 2008

New Orleans, LA

May 16, 2008


Presented at aapor 2008 new orleans la may 16 2008

Thomas M. Guterbock

[email protected]

James M. Ellis

[email protected]

Abdoulaye Diop

[email protected]

Kien Le

[email protected]

John Lee Holmes

[email protected]

CSR—University of Virginia

www.virginia.edu/surveys

. . . A unit of the Weldon Cooper Center for Public Service


The research problem rdd under threat are there good alternatives

The Research Problem:RDD under threat Are there good alternatives?


Rdd under threat

RDD under threat

Random Digit Dialing involves a certain degree of inefficiency

Costs of this extra effort justified by completeness of coverage (at least until recently)

Recent trends raising these costs:

decreasing density of working numbers,

increasing rates of non-contact,

and rising rates of refusal

Advent of cellular phone only households diminishes completeness of its coverage

4


A new norm

A “New Norm?”

  • Dual-frame “RDD+Cell” has arisen in response to these challenges

    • traditional list-assisted RDD sample with RDD of working cellphone exchanges.

  • To screen or not to screen?

    • And, if not, how to weight?

  • But other dual frames may also be worth exploring . . .


Proposed alternative ewp cell

Proposed Alternative: EWP+Cell

  • EWP+Cell =

    • “Electronic White Pages” + Cell Phone RDD

  • Promises considerably greater efficiency and cost savings over RDD+Cell

    • especially for specific, small geographic regions

    • or areas not co-extensive with any set of telephone Area Codes.

  • EWP+Cell fails to cover: unlisted landline households that have no cell phone

    • We will examine: How big a problem is that?


Data source 2006 national health interview survey

Data Source:2006 National Health Interview Survey

permits estimations of the size of . . .

  • the non-covered segment

  • demographic characteristics

  • health characteristics

  • degree of coverage bias


What did we find a preview

What did we find?A Preview

  • Surprisingly little coverage bias to be expected from EWP+Cell

  • Potential cost savings from EWP+Cell compared to RDD+Cell


A brief review of the research

A brief review of the research

Not much literature or research compares directory-listed samples with list-assisted, landline RDD samples

Consequently, the degree and nature of the differences between listed and unlisted households is not established.

9


Older studies before 2002

Older studies (before 2002)

Most found only slight differences in substantive results between EWP and RDD frame samples

Some efficiency gains in smaller geographic areas

Some differences in demographics noticed

And in 2007, Zogby announced plans to rely on EWP over RDD phone samples, citing lack of substantive differences in results.

10


Recent studies on ewp vs rdd substantial differences shown

Recent studies on EWP vs RDD: Substantial differences shown

  • Unlisted rates are higher for:

    • Blacks, Hispanics

    • Lower income

    • Renters

    • Single people

    • See: Guterbock, Diop

    • and Holian (2007)


From 3 segments to 5

From 3 segments to 5


The universe of u s telephone households 2006

The universe of U.S. telephone households, 2006


Rdd samples cover all landline households listed or not

RDD samples cover all landline households, listed or not

RDD

Cell-phone-

only households

are excluded


Cell phone samples include some that are also in the rdd frame

Cell phone samples include some that are also in the RDD frame

Landline-

only

households

are excluded

Cell phones


Rdd and cell samples overlap yield complete coverage

RDD and Cell samples overlap,yield complete coverage

RDD

CELL + LANDLINE

52.0%

CELL ONLY

16.6%

LANDLINE

ONLY

31.4%

Cell phones

All percentages are from 2006 NHIS data.


We need also to consider listedness

We need also to consider: listedness

  • Some landlines are listed in the residential directory or Electronic White Pages [EWP]

    • LLL = Listed Landline

  • Some landline households are unlisted

    • ULL = unlisted landline

  • LLL and ULL may or may not also have a cell phone in the household

  • Cell phones are unlisted by definition

  • Result: five segments of the telephone universe . . .


Five telephone segments

Five telephone segments

2

CELL + ULL

17.7%

3

ULL ONLY

14.2%

1

CELL ONLY

16.6%

4

CELL + LLL

34.3%

5

LLL ONLY

17.2%

See table I

All percentages are from 2006 NHIS data.


Five segments differ sometimes sharply

Five segments differ, sometimes sharply


Segments differ on key demographics

Segments differ on key demographics

See table II

All percentages are from 2006 NHIS data.


Segments differ on key health questions

Segments differ on key health questions

See table II

All percentages are from 2006 NHIS data.


Our analysis deals with three sampling frames

Our analysis deals withthree sampling frames:

  • EWP

  • List-assisted (landline) RDD

  • Cell phone RDD


We examine 4 sampling designs

We examine 4 sampling designs:

  • 2 Single frame designs:

    • EWP only

    • Landline RDD


We examine 4 sampling designs1

We examine 4 sampling designs

  • 2 Dual Frame designs:

    • EWP+Cell

    • RDD+Cell


Three design contrasts

Three design contrasts

RDD+Cell is the base for all comparisons

  • It includes the full universe of phone HH

    We will compute coverage bias for each contrast:

  • EWP vs. RDD+Cell

  • RDD vs. RDD+Cell

  • EWP+Cell vs. RDD+Cell


  • Formula for coverage bias

    Formula for coverage bias

    Ῡ= mean for full population

    ῩC = mean for covered cases

    ῩU = mean for cases not covered

    U = cases not covered

    N = all cases


    Contrast i ewp vs rdd cell telephone samples

    Contrast I: EWP vs. RDD+Cell telephone samples


    The universe of u s telephone households

    The universe of U.S. telephone households


    Ewp sample excludes unlisted landline and cell only

    EWP sample excludes unlisted landline and cell-only

    2

    EXCLUDED

    CELL + ULL

    17.7%

    ῩU

    3

    EXCLUDED

    ULL ONLY

    14.2%

    1

    EXCLUDED

    CELL ONLY

    16.6%

    4

    CELL + LLL

    34.3%

    5

    LLL ONLY

    17.2%

    U/N = .485

    EWP

    All listed landline phones

    ῩC


    Coverage bias table ewp vs rdd cell

    Coverage bias table:EWP vs. RDD+Cell

    See table IV

    All percentages are from 2006 NHIS data.


    Presented at aapor 2008 new orleans la may 16 2008

    Coverage bias table:EWP vs. RDD+Cell

    See table IV

    All percentages are from 2006 NHIS data.


    Contrast ii rdd vs rdd cell only telephone samples

    Contrast II: RDD vs. RDD+Cell only telephone samples


    Rdd samples cover all landline households listed or not1

    RDD samples cover all landline households, listed or not

    RDD

    ῩC

    ῩU


    Rdd fails to cover 16 6

    RDD fails to cover 16.6%

    RDD

    CELL + LANDLINE

    52.0%

    CELL ONLY

    16.6%

    LANDLINE

    ONLY

    31.4%

    U/N = .166

    Cell phones

    All percentages are from 2006 NHIS data.


    Coverage bias table rdd vs rdd cell

    Coverage bias table:RDD vs. RDD+Cell

    See table V

    All percentages are from 2006 NHIS data.


    Presented at aapor 2008 new orleans la may 16 2008

    Coverage bias table:RDD vs. RDD+Cell

    See table V

    All percentages are from 2006 NHIS data.


    Contrast iii ewp cell vs rdd cell telephone samples

    Contrast III: EWP+Cellvs. RDD+Cell telephone samples


    Rdd cell covers all phone households

    RDD+Cell covers all phone households

    RDD

    CELL + LANDLINE

    52.0%

    CELL ONLY

    16.6%

    LANDLINE

    ONLY

    31.4%

    Cell phones


    Ewp cell sample design

    EWP + Cell Sample Design

    EXCLUDES

    ULL-

    ONLY

    Cell

    EWP

    All listed landline phones


    Ewp cell excludes ull only households

    EWP + Cell excludes ULL-only households

    EXCLUDED:

    3

    ULL ONLY

    14.2%

    2

    CELL + ULL

    17.7%

    U/N = .142

    1

    CELL ONLY

    16.6%

    ῩU

    4

    CELL + LLL

    34.3%

    5

    LLL ONLY

    17.2%

    ῩC


    Coverage bias table ewp cell vs rdd cell

    Coverage bias table: EWP+Cell vs. RDD+Cell

    See table VI

    All percentages are from 2006 NHIS data.


    Coverage bias table ewp cell vs rdd cell1

    Coverage bias table: EWP+Cell vs. RDD+Cell

    See table VI

    All percentages are from 2006 NHIS data.


    Summary of 3 contrasts

    Summary of 3 contrasts


    3 contrasts 2006 estimates c

    3 contrasts: 2006 estimates (ῩC )

    See table VII


    3 contrasts 2006 raw bias c

    3 contrasts: 2006 raw bias (ῩC -Ῡ)

    See table VIII


    3 contrasts 2006 percent bias

    3 contrasts: 2006 percent bias

    See table IX


    Changes in coverage nhis 2003 2006

    Changes in CoverageNHIS 2003 - 2006


    Changes in telephone status over time

    unlisted landline only

    cell phone only

    Changes in telephone status over time


    Changes in percent bias over time rdd vs rdd cell nhis data

    Changes in percent bias over time: RDD vs. RDD+Cell (NHIS data)


    Changes in percent bias over time ewp cell vs rdd cell nhis data

    Changes in percent bias over time: EWP+Cell vs. RDD+Cell (NHIS data)


    Changes in percent bias over time rdd vs rdd cell nhis data1

    Changes in percent bias over time: RDD vs. RDD+Cell (NHIS data)


    Changes in percent bias over time ewp cell vs rdd cell nhis data1

    Changes in percent bias over time: EWP+Cell vs. RDD+Cell (NHIS data)

    52


    Cost comparisons

    Cost comparisons


    Cost comparisons1

    Cost comparisons

    *CPH= completions per hour


    Cost comparisons data collection only

    Cost comparisons(data collection only)


    Conclusions

    Conclusions

    EWP+Cell omits ULL–onlies, but:

    These aren’t particularly untypical

    They are not numerous

    Their numbers are declining

    EWP+Cell includes unlisteds (who have cell phones) thus avoiding some bias from EWP

    EWP+Cell includes cell–onlies, offsetting bias from omitting unlisted HH

    Unlisted are somewhat similar to the cell–onlies.

    56


    Conclusions1

    Conclusions

    • We propose EWP+Cell as a cost effective sampling solution that appears to offer good coverage across age, race, most demographics, and key health indicators

    • It offers a significant cost advantage over the ‘new norm’ (RDD+Cell), especially if:

      • geographic area to be studied is small

      • target population is hard to find (requiring screener calls)


    Cautions caveats

    Cautions & caveats

    • We have not offered a direct, experimental field test of EWP+Cell sampling in contrast with RDD+Cell

      • But we have several such experiments planned in our 2008 local surveys

    • Nobody knows the proper weights for combining the two sample frames

      • but these are also unknown for local studies that use “the new norm” –RDD+Cell

    • 2008 phone segments could differ from 2006 more than we think


    More cautions

    More cautions . . .

    • Our analysis assumes that non-response and measurement errors are the same in realized samples from each telephone segment

      • That is, we have considered coverage error only

    • Even if the proportion of excluded cases (ULL-onlies) is declining, coverage error may not decrease, because:

      as changes, so can .


    Planned tests

    Planned tests

    • We are currently conducting a county-wide citizen survey in Prince William County, VA, that features

      • A 10% cell phone component (unscreened)

      • The balance of completions split:

        • 45% RDD sample

        • 45% EWP sample

    • We will be able to compare directly:

      • EWP+Cell vs. RDD+Cell

      • Coverage bias, productivity, and costs


    Presented at aapor 2008 new orleans la may 16 2008

    Who Needs RDD? Combining Directory Listings with Cell Phone Exchanges for an Alternative Sampling Frame

    e-mail: [email protected]

    Presented at

    AAPOR 2008

    New Orleans, LA

    May 16, 2008


  • Login