Mixed mode data collection recent developments at statistics netherlands
This presentation is the property of its rightful owner.
Sponsored Links
1 / 13

Mixed-mode data collection Recent developments at Statistics Netherlands PowerPoint PPT Presentation


  • 99 Views
  • Uploaded on
  • Presentation posted in: General

Mixed-mode data collection Recent developments at Statistics Netherlands. Gabriël van Dam. Outline. Background and motivation Quality and costs Redesigns Mode effects Conclusions and discussion. Background Quality & costs Redesigns Mode effects Discussion.

Download Presentation

Mixed-mode data collection Recent developments at Statistics Netherlands

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Mixed mode data collection recent developments at statistics netherlands

Mixed-mode data collectionRecent developments at Statistics Netherlands

Gabriël van Dam


Outline

Outline

  • Background and motivation

  • Quality and costs

  • Redesigns

  • Mode effects

  • Conclusions and discussion

Seminar on Statistical Data Collection – September 26, 2013


Developments at stat netherlands

Background

Quality & costs

Redesigns

Mode effects

Discussion

Developments at Stat Netherlands

Generally, focus on costs:

Project Redesign of Social Surveys. 2007 – 2012

Focus on quality:

Project Mode Effects in Social Surveys, 2010 – 2012

Adaptive survey designs:

Studies for CVS and LFS, 2011 - 2013

Seminar on Statistical Data Collection – September 26, 2013


Focus on costs

Background

Quality & costs

Redesigns

Mode effects

Discussion

Focus on costs

At Statistics Netherlands, five main questions:

  • What are response rates for web and Mixed-mode designs?

  • Can we do Mixed-mode designs with web?

  • What should be lengths of questionnaires and how to transform complex survey items?

  • Are new statistics comparable and stable?

  • What about undercoverage of web?

Seminar on Statistical Data Collection – September 26, 2013


Redesign of social surveys

Background

Quality & costs

Redesigns

Mode effects

Discussion

Redesign of Social Surveys

Conclusions to main redesign questions

Response rates:

  • Web cannot be single mode

  • MM designs have response rates and representativeness similar to CAPI

  • Web recruitment rates lower than CATI/CAPI

    Implementation:

  • Major changes to administration systems

  • Complex monitoring of sample cases

  • Data collection period was lengthened

Seminar on Statistical Data Collection – September 26, 2013


Redesign of social surveys1

Background

Quality & costs

Redesigns

Mode effects

Discussion

Redesign of Social Surveys

Questionnaires:

  • Questionnaires had to be bisected

  • Complex items simplified or replaced by registries

    Comparability and stability:

  • Parallel runs conducted for all redesigns

  • Quality reports before and after redesigns in order to support explanation of method effects

  • Web shows unstable response rates

  • Large mode effects are possible

    Coverage web:

  • Web access rate at home is 90%

  • Undercoverage based on age, income, ethnicity and urbanization degree

Seminar on Statistical Data Collection – September 26, 2013


Context lfs and cvs redesigns

Background

Quality & costs

Redesigns

Mode effects

Discussion

Context – LFS and CVS redesigns

LFS LFS LFS

<2010 2010 – 2012 >2012

web

CAPI

CATI

CAPI

CATI

CAPI

CVS (SN and public) CVS

<2011 >2012

web+

web+

web + paper

CATI

CAPI

CATI

Seminar on Statistical Data Collection – September 26, 2013


Mode effect components

Background

Quality & costs

Redesigns

Mode effects

Discussion

Mode effect components

Data collection steps:

Persons need to be reached (coverage)

Persons need to respond (response)

Persons need to provide valid answers (measurement)

Total relative mode effect (telephone – face-to-face):

Mtel(y) = ytel,tel,tel – ytot,f2f,f2f

coverage

response

measurement

Seminar on Statistical Data Collection – September 26, 2013


Mode effects conclusions

Background

Quality & costs

Redesigns

Mode effects

Discussion

Mode effects - conclusions

CVS:

  • Telephone most optimistic and web most pessimistic

  • Measurement effect is dominant effect

  • Mode effect very strong on victimization topics

  • Questionnaire needs redesign

    LFS:

  • Mode effect is mix of effects

  • Selection effect can be explained using register variables

    General:

  • Mode-specific measurement effects will lead to method effects in redesigns

Seminar on Statistical Data Collection – September 26, 2013


Mode effects implications

Background

Quality & costs

Redesigns

Mode effects

Discussion

Mode effects - implications

CVS:

  • Weighting only removes a small part of the mode effect

  • Mode effects must be avoided by design and/or must be stabilized by calibration to fixed mode totals

    LFS:

  • Weighting removes a large part of the mode effect

  • Remaining mode effect is difference in measurement

Seminar on Statistical Data Collection – September 26, 2013


Conclusions

Background

Quality & costs

Redesigns

Mode effects

Discussion

Conclusions

  • Web cannot be used as a single mode for social surveys and lead to MM designs

  • MM surveys including web are feasible after some learning period but require investments in infrastructure, monitoring and pilot studies

  • Inclusion of web as a survey mode implies a redesign of survey questionnaires

  • Mode effects between web and interviewer modes can be large

Seminar on Statistical Data Collection – September 26, 2013


Recommendations

Background

Quality & costs

Redesigns

Mode effects

Discussion

Recommendations

Reduce, avoid and adjust:

  • Reduce measurement effects by careful questionnaire design

  • Avoid contrasting modes and assign resources to subpopulations that show strongest mode effects

  • Adjust for method effects by stabilizing or calibrating the distribution of respondents over modes

Seminar on Statistical Data Collection – September 26, 2013


Discussion

Background

Quality & costs

Redesigns

Mode effects

Discussion

Discussion

  • Web response rates are still much lower than other modes. Increase of rates should be possible?

  • How to deal with household data collection in web surveys (in terms of self and proxy reporting)?

  • How to increase web recruitment rates for subsequent waves?

  • How to stabilize statistics in MM surveys including web?

    Questions? Email: [email protected]

Seminar on Statistical Data Collection – September 26, 2013


  • Login