E&I of administrative data used for producing business statistics

1. E&I of administrative data used for producing business statistics Vera Costa, Frances Krsinich, Rudi Van der Mescht 2008 UNECE Work Session on Statistical Data Editing

2. Overview • Introduction • Electronic Card Transaction Data • Background • E&I • Longitudinal Business Database • Background • E&I • Future work • Questions

3. Introduction • E&I methodologies for: • private vs government administrative data • unit record vs aggregated level data

4. Electronic Card Transaction Data • Background • Concept • Data and providers • Feasibility project - comparison with Retail Trade Survey estimates • Consultation process

5. Electronic Card Transaction Data(cont.) • Background (cont.) • Current situation – series published (actuals, trend and seasonally adjusted) • Total Electronic Card Transactions • Retail Electronic Card Transactions • Core Retail Electronic Card Transactions

6. Electronic Card Transaction Data(cont.) • E&I • Data characteristics and implications • Possible issue: incomplete data provided • Problem identification • By provider • By Statistics NZ • Problem solving • Preferred approach – contact the provider • Alternative approach – imputation

7. Electronic Card Transaction Data(cont.) • E&I (cont.) • Imputation related issues • Methodology • Pro-rating • Time series methods • Total forecasted value lower than actual value from other provider(s) • Big merchant changing providers • Metadata and system requirements

8. Electronic Card Transaction Data(cont.) • E&I (cont.) • Current situation • Experimental series published monthly • Data quality assessment • Analysis of the data received • Liaison with the providers • Media monitoring

9. Longitudinal Business Database • Background • Data and providers • Sales and purchases from Goods and Services Tax (GST returns) • Financial performance and position variables (IR10 tax forms) • Salaries and wages, and employee counts (Pay-As-You-Earn returns – PAYE)

10. Longitudinal Business Database (cont.) • Background (cont.) • Data requirements • Approach used: advantages / disadvantages • Flag indicators

11. Longitudinal Business Database (cont.) • Background (cont.) • Current situation • Data from 2000 to 2005 • Around 800,000 enterprises in the database • Many related variables – auxiliary information for E&I

12. Longitudinal Business Database (cont.) • Data characteristics • Editing process

13. Longitudinal Business Database (cont.) • Imputation process • Project’s first year • Methods used • Mean • Multiple • Donor (nearest neighbour) • Number of variables to be imputed • Donors characteristics – bias introduction • Matching variables – PAYE – data issues • Evaluation – simulated ‘missingness’

14. Longitudinal Business Database (cont.) • Imputation process (cont.) • Project’s second year • Changes introduced • More use of existing data • Interpolation • Carry forward • Data subsetted by “lifespan” • Additional matching variables (IR10 and GST)

15. Longitudinal Business Database (cont.)

16. Longitudinal Business Database (cont.) • Future work • Investigation of the most appropriate imputation method – longitudinal admin data • If donor imputation is chosen: • Use of donors to impute movements rather than levels • More use of past / future values in the longitudinal record • Treat large non-respondents separately

17. Questions • From the audience? • From us: • Imputation methods when aggregated administrative data is received? • Donor imputation for movements?

18. Contacts • Vera Costa • Vera.Costa@stats.govt.nz • Frances Krsinich • Frances.Krsinich@stats.govt.nz • Rudi Van der Mescht • Rudi.VanderMescht@stats.govt.nz