Exploring new methods for household survey designs at Statistics New Zealand

1. Exploring new methods for household survey designs at Statistics New Zealand Steven Johnston Statistical Methods

2. Overview • Recent work on household survey sample designs, to make use of new data sources and new methods • Administrative data for social statistics • Use of internet in the Census and surveys

3. Background • 5-yearly Census of Population and Dwellings • Three main on-going social surveys • Household Labour Force Survey • Household Economic Survey • General Social Survey • Additional topics as add-on modules

4. Survey sample designs use an areal frame • Historically no list of dwellings to sample from • Two stage sample design where Primary Sampling Units (PSUs – small areas of about 60 dwellings) are first selected, followed by a sample of dwellings • PSUs are stratified by location and socio-economic characteristics

5. Following each 5-yearly Census, we update the frame • Change PSU boundaries so that PSUs remain even in size • Re-stratify PSUs based on up-to-date characteristics

6. This year, we’re also reviewing other aspects of our methodology… • Can we make use of address lists and move to sampling addresses directly? • How can we use geospatial tools to improve our PSU pattern (and also do less work)? • Should we use a dual frame method to survey the Maori population more effectively?

7. Should we sample addresses rather than PSUs? • Government-wide initiative is expected to improve the quality of address lists • Can we use this to sample addresses directly? • Main trade-off: • Increased collection cost vs more efficient design (reduced design effect) • Collection costs would be too high in rural areas

8. Move from this ….

9. to this …

10. Results • Change in collection costs lower than expected (10-15% increase only). But a lot of uncertainty around this. • DEFF and cost about even unless collection mode changes significantly • Main benefit would be targeting key subgroups more easily (e.g. link address register to Census or tax information) • Decided against unclustering at this stage • Mainly because of uncertainty in the quality of the address register

11. How can we make use of address lists? • Need to understand their quality • Will be comparing address lists to Census addresses later this year • This will inform strategy for listing addresses within the PSUs we select for a survey • Hope to reduce field listing activity by >80% • But main goal is an improvement in listing quality

12. Making use of GeoSpatial tools • Previously we have ‘re-formed’ PSUs manually • List PSUs which are outside an acceptable range • Have staff manually look at maps and identify the best way to change the PSU (e.g. split into two or merge) • Took 2 staff about 4 months for the last reformation • This year planning to use ArcGIS to do this automatically

13. Results are very promising • Identified that almost 10% of PSUs have segments that do not share a connecting road – aim to fix most of this • Setting the procedures up in advance means we can reduce the time between getting Census data and being able to select a new sample • Large reduction in the variation in PSU sizes

14. Targeting Maori more effectively • Maori are the indigenous population of New Zealand • 14% of the population • Key group of interest for policy and research • Currently we over-sample areas with high density • Gives only small gains, increasing the proportion of Maori sampled to 16%

15. Ministry of Health’s NZ Health Survey (NZHS) recently started using the Electoral Roll to target addresses likely to contain Maori • The NZ Electoral Roll: • covers > 90% of Adults aged 18 years and older • asks if people are of Maori decent • address information is actively maintained by linking to administrative data sources • up-date campaigns before 3-yearly elections

17. Reviewed NZHS results to see if Statistics NZ should use the same approach • Gains weren’t as big as expected • By sacrificing a 5% increase in SEs at national level… • … can reduce Maori SEs by 10% using the Electoral Roll • … or reduce Maori SEs by 8% by doing areal targeting better • Include the number of Maori in PPS size measure

19. Administrative data • Linked Employer-Employee Database (LEED) • Personal tax data linked with information about employers from Statistics NZ’s Business Frame • Has been further developed in the last 2 years…

21. IDI+ • New project to bring together person data held by many Government agencies, at Statistics NZ • Linked by us • Made available to analysts across Government, in Statistics NZ’s secure on-site Datalab (or remote access?) • Powerful new source of information for social and population statistics • Perhaps (eventually) an alternative to Census?

22. Internet surveys • 2006 Census = 11% used internet form • 2013 Census = 34% • Live test in one district = 65% • Current focus on business surveys • Then testing internet for Labour Force Survey

23. RealMe • New initiative to enable people to create a verified digital identity to access Government services online • Goal by 2017 to have at least 70% of citizens’ interactions with Government on-line

24. Vision of the future? • Database of linked administrative data about people from across Government • Linked to a spatially-enabled address register • Internet surveys to collect additional social data

25. Thank you • steven.johnston@stats.govt.nz