mixed mode data collection recent developments at statistics netherlands
Download
Skip this Video
Download Presentation
Mixed-mode data collection Recent developments at Statistics Netherlands

Loading in 2 Seconds...

play fullscreen
1 / 13

Mixed-mode data collection Recent developments at Statistics Netherlands - PowerPoint PPT Presentation


  • 134 Views
  • Uploaded on

Mixed-mode data collection Recent developments at Statistics Netherlands. Gabriël van Dam. Outline. Background and motivation Quality and costs Redesigns Mode effects Conclusions and discussion. Background Quality & costs Redesigns Mode effects Discussion.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Mixed-mode data collection Recent developments at Statistics Netherlands' - manchu


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
outline
Outline
  • Background and motivation
  • Quality and costs
  • Redesigns
  • Mode effects
  • Conclusions and discussion

Seminar on Statistical Data Collection – September 26, 2013

developments at stat netherlands

Background

Quality & costs

Redesigns

Mode effects

Discussion

Developments at Stat Netherlands

Generally, focus on costs:

Project Redesign of Social Surveys. 2007 – 2012

Focus on quality:

Project Mode Effects in Social Surveys, 2010 – 2012

Adaptive survey designs:

Studies for CVS and LFS, 2011 - 2013

Seminar on Statistical Data Collection – September 26, 2013

focus on costs

Background

Quality & costs

Redesigns

Mode effects

Discussion

Focus on costs

At Statistics Netherlands, five main questions:

  • What are response rates for web and Mixed-mode designs?
  • Can we do Mixed-mode designs with web?
  • What should be lengths of questionnaires and how to transform complex survey items?
  • Are new statistics comparable and stable?
  • What about undercoverage of web?

Seminar on Statistical Data Collection – September 26, 2013

redesign of social surveys

Background

Quality & costs

Redesigns

Mode effects

Discussion

Redesign of Social Surveys

Conclusions to main redesign questions

Response rates:

  • Web cannot be single mode
  • MM designs have response rates and representativeness similar to CAPI
  • Web recruitment rates lower than CATI/CAPI

Implementation:

  • Major changes to administration systems
  • Complex monitoring of sample cases
  • Data collection period was lengthened

Seminar on Statistical Data Collection – September 26, 2013

redesign of social surveys1

Background

Quality & costs

Redesigns

Mode effects

Discussion

Redesign of Social Surveys

Questionnaires:

  • Questionnaires had to be bisected
  • Complex items simplified or replaced by registries

Comparability and stability:

  • Parallel runs conducted for all redesigns
  • Quality reports before and after redesigns in order to support explanation of method effects
  • Web shows unstable response rates
  • Large mode effects are possible

Coverage web:

  • Web access rate at home is 90%
  • Undercoverage based on age, income, ethnicity and urbanization degree

Seminar on Statistical Data Collection – September 26, 2013

context lfs and cvs redesigns

Background

Quality & costs

Redesigns

Mode effects

Discussion

Context – LFS and CVS redesigns

LFS LFS LFS

<2010 2010 – 2012 >2012

web

CAPI

CATI

CAPI

CATI

CAPI

CVS (SN and public) CVS

<2011 >2012

web+

web+

web + paper

CATI

CAPI

CATI

Seminar on Statistical Data Collection – September 26, 2013

mode effect components

Background

Quality & costs

Redesigns

Mode effects

Discussion

Mode effect components

Data collection steps:

Persons need to be reached (coverage)

Persons need to respond (response)

Persons need to provide valid answers (measurement)

Total relative mode effect (telephone – face-to-face):

Mtel(y) = ytel,tel,tel – ytot,f2f,f2f

coverage

response

measurement

Seminar on Statistical Data Collection – September 26, 2013

mode effects conclusions

Background

Quality & costs

Redesigns

Mode effects

Discussion

Mode effects - conclusions

CVS:

  • Telephone most optimistic and web most pessimistic
  • Measurement effect is dominant effect
  • Mode effect very strong on victimization topics
  • Questionnaire needs redesign

LFS:

  • Mode effect is mix of effects
  • Selection effect can be explained using register variables

General:

  • Mode-specific measurement effects will lead to method effects in redesigns

Seminar on Statistical Data Collection – September 26, 2013

mode effects implications

Background

Quality & costs

Redesigns

Mode effects

Discussion

Mode effects - implications

CVS:

  • Weighting only removes a small part of the mode effect
  • Mode effects must be avoided by design and/or must be stabilized by calibration to fixed mode totals

LFS:

  • Weighting removes a large part of the mode effect
  • Remaining mode effect is difference in measurement

Seminar on Statistical Data Collection – September 26, 2013

conclusions

Background

Quality & costs

Redesigns

Mode effects

Discussion

Conclusions
  • Web cannot be used as a single mode for social surveys and lead to MM designs
  • MM surveys including web are feasible after some learning period but require investments in infrastructure, monitoring and pilot studies
  • Inclusion of web as a survey mode implies a redesign of survey questionnaires
  • Mode effects between web and interviewer modes can be large

Seminar on Statistical Data Collection – September 26, 2013

recommendations

Background

Quality & costs

Redesigns

Mode effects

Discussion

Recommendations

Reduce, avoid and adjust:

  • Reduce measurement effects by careful questionnaire design
  • Avoid contrasting modes and assign resources to subpopulations that show strongest mode effects
  • Adjust for method effects by stabilizing or calibrating the distribution of respondents over modes

Seminar on Statistical Data Collection – September 26, 2013

discussion

Background

Quality & costs

Redesigns

Mode effects

Discussion

Discussion
  • Web response rates are still much lower than other modes. Increase of rates should be possible?
  • How to deal with household data collection in web surveys (in terms of self and proxy reporting)?
  • How to increase web recruitment rates for subsequent waves?
  • How to stabilize statistics in MM surveys including web?

Questions? Email: [email protected]

Seminar on Statistical Data Collection – September 26, 2013

ad