Mixed-Mode Approaches in the Generations and Gender Survey Past Experience and Future Expectations

1. Mixed-Mode Approaches in the Generations and Gender SurveyPast Experience and Future Expectations Aat Liefbroer Netherlands Interdisciplinary Demographic Institute

2. Overview Overview of the presentation • GGP: what is it? • Past experience • 3rd wave of Dutch GGP 2010-2011 • Future expectations • Pilot 4th wave in Slovenia 2011 • Blueprint for GGP 2015 • Conclusions

3. Overview Aim of the GGP • To advance our knowledge on intergenerational and gender relationships, with a focus on understanding gender inequalities and generational differences • Internationally comparable data on relationships between partners and between parents and adult children across Western societies • Main vehicles: Generations and Gender Survey (GGS) and Contextual Database

4. Overview Characteristics of the GGS • Broad age-range (18-79), because understanding demographic challenges such as population ageing asks for a focus on both young and old people • Large-scale surveys on population challenges (N≈10,000) to allow in-depth analysis • Panel design to allow for better causal analysis and studying processes of adaptation to change • Cross-national to allow for examining the influence of the social context (including the policy context) • Theory-driven questionnaire design • Contextual macro-level database to allow for multi-level analyses

5. Overview GGP Countries • Wave 1 • 18 countries (data for 12 countries currently available) • Wave 2 • Bulgaria, Netherlands, Australia, Germany, France, Hungary, Italy, Russian Federation, Georgia (Bulgaria and Germany to be released this year)

6. Overview Challenges for the future of the GGP • Increasing the number of participating countries • Extending the data-collection beyond three waves • Reducing fieldwork costs • Application of mixed mode design?

7. 3rd wave Dutch GGP Dutch GGP • 1st wave 2004 ≈ 8,200 respondents, CAPI • 2nd wave 2007 ≈ 6,200 respondents, CAPI • 3rd wave 2010 ≈ 4,300 respondents, mixed mode • Response rate in 3rd wave: 72% of wave 2 participants • Costs per respondent in 2nd wave: ≈ €195 • Costs per respondent in 3rd wave: ≈ €115

8. 3rd wave Dutch GGP Distribution by mode

9. 3rd wave Dutch GGP Mode distribution by age

10. 3rd wave Dutch GGP Mode distribution by educational attainment

11. 3rd wave Dutch GGP Missing income info by mode • Percentage that does not mention earnings from labour

12. 3rd wave Dutch GGP Disclosure • Mean score on loneliness scale by mode (with and without controls)

13. 3rd wave Dutch GGP Attrition information (I) • Percentage that gives permission to contact their partner to fill out a questionnaire

14. 3rd wave Dutch GGP Attrition information (II) • Percentage that gives permission to link data to register data from Statistics Netherlands

15. 3rd wave Dutch GGP Attrition information (III) • Percentage that gives permission to be recontacted for the fourth wave

16. 3rd wave Dutch GGP Conclusions • WEB more popular than expected • Missing values on WEB limited • Small, but systematic mode effects in disclosure • WEB respondents are less willing to give permission for additional or future involvement

17. Pilot Slovenia Pilot 4th wave Slovenia 2011 • Participating institutions • University of Ljubljana (Lozar-Manfreda, Petrič) • University Utrecht (Hox, De Leeuw) • NIDI (Kveder, Liefbroer) • Study 1: comparison of mode effects • Study 2: comparison of mixed mode systems

18. Pilot Slovenia Comparison of mode effects • Aim: testing new pilot and examination of potential mode effects • Sample: regular participants in commercial web panel, randomly assigned to different modes • Examination of • Item non-response • Scalability of item sets • Distributional characteristics • Length of interview • Evaluation of interview

19. Pilot Slovenia Comparison of mode effects • Item non-response only slightly higher in web, with one clear exception….

20. Pilot Slovenia Income • Percentage of refusals and ‘do not know’ on question on household income

21. Pilot Slovenia Comparison of mode effects • Item non-response only slightly higher in web, with one clear exception…. • Scalability of item sets still has to be analysed • First analyses of distributional characteristics suggest • More disclosure on social desirable and difficult questions • Slightly higher variance in responses

22. Pilot Slovenia Example • Percentage answering ‘yes’ to the question ‘Over the past 12 months, have you thought about breaking up your relationship?’

23. Pilot Slovenia Comparison of mode effects • Item non-response only slightly higher in web, with one clear exception…. • Scalability of item sets still has to be analysed • First analyses of distributional characteristics suggest • More disclosure on social desirable and difficult questions • Slightly higher variance in responses • Interview took somewhat longer in CATI mode • Evaluation of interview most negative in CATI mode

24. Pilot Slovenia Duration of survey

25. Pilot Slovenia Subjective length of survey • Percentage stating ‘yes’ or ‘definitely yes’ to question whether the interview was judged too long

26. Pilot Slovenia Comparison of mixed-mode systems (I) • Aim: testing of alternative mixed-mode systems • Sample: two-stage random sample from the population register • Examination of • Response rate of different mixed-mode systems • Costs of different mixed mode systems • Evaluation of modes by a random sample

27. Pilot Slovenia Comparison of mixed-mode systems (II) • Comparison of four ‘systems’ • CAPI → WEB → CATI (no incentive offered) • CATI → WEB → CAPI (no incentive offered) • WEB → CATI → CAPI (no incentive offered) • WEB → CATI → CAPI (incentive of € 5 offered) • Difference in response rate across systems • Difference in costs across systems

28. Blueprint GGP 2015 Blueprint GGP 2015 • Blueprint for GGP 2015 should be ready by the end of 2012 • Questionnaire GGP 2015 • Sampling design • Set of fieldwork rules • Whether to opt for mixed-modes, and if so when? • Is it an option in the first wave (or in the first wave with a refreshment sample), or in later waves only? • What kind of mixed-mode system to prescribe?

29. Conclusion Conclusions • Overall, WEB seems to perform quite well in terms of answering patterns (item nonresponse, scalability and distribution of responses) • WEB leads to very considerable cost reductions • The big issue that is not yet clear is whether attrition is negatively affected • Use of WEB in a first wave seems questionable (but pilot will give partial answers) • Use in later waves could lead to higher dropout in future waves as a result of the legal requirement to ask for permission for re-contacting

30. Thank you for your attention!

31. Overview Topics included • Provision of care to older adults • Successful ageing • Economic participation of women and pre-pensioners • Realisation of fertility intentions • Balancing work and family life • Gender equality in household and childcare labour

32. Members of Consortium Board • Netherlands Interdisciplinary Demographic Institute (NL) • Institut National d’ÉtudesDémographiques (FR) • Bocconi University (IT) • Statistics Norway (NO) • Demographic Research Institute (HU) • Norwegian Social Research (NO) • University of Ljubljana (SL) • Utrecht University (NL) • University of York (GB) • Max-Planck Institute for Demographic Research (DE) • Erasmus University Rotterdam (NL) • Population Unit of the UN Economic Commission for Europe