The Quality of French census

1. The Quality of French census

2. An « old » but good paper… • Written en 2008 by Olivier Lefebvre and Michel Cezard • Three parts : • A complete presentation of the new french population census method • Compliance on • the quality criteria of the UN (individual enumeration, universality,….) • the criteria shortlisted by the EuroStat code of best practices (relevance, accuracy,…) • The management of the census quality

3. II - In line with quality criteria • The essential characteristics of the census (defined by the UN) : • Individual listing • Universality • Simultaneity • Capacity to supply data on small fields • Regularity

4. Individual listing • Data for each individual and each household interviewed in the census • Organised in a « detail » file allowing for all kinds of comparisons • Sampling rate sufficient for all kinds of cross-tabulations

5. Capacity to supply data on small fields • Stems from both universality and individual listing • Garanteed by the sampling rate (100% in the small communes and 40% in the large ones) • The census will supply data on all the communes and on the « IRIS », districts of about 2000 inhabitants

6. Regularity • Criteria met by construction because one supplies annual data

7. In line with quality criteria (2) • Criteria adopted by Eurostat • Relevance • Accuracy and reliability • Timeliness and punctuality • Accessibility and clarity • Comparability • Coherence

8. Accuracy and reliability • An imprecision linked to the sample survey, but of low amplitude and lower than the one resulting from the «ageing » of data • Data collection, control and computation methods which considerably reduce the omission bias

9. Timeliness and punctuality • Recent data • permanently 2 to 3 years for detailed data against 18 months to 10 years in the former system • results of annual surveys available at the beginning of the following year and integrated to the provisional estimates (at the regional and national level) • Punctuality : • Legal requirement (legal populations of the communes) • It guarantees the permanence of the system

10. Accessibility and clarity • Accessibility guaranteed by resorting to the internet as medium for dissemination • Systematic feed-back to the communes, with the possibility to re-disseminate this data • Clarity : important pedagogical and documentation work in the website.

11. III - The management of the census quality • General principles • control quality step by step • A continuous evaluation approach • Exemples : • Sampling frames • Data collection • Data processing • Dissemination

13. Evaluation Campaign Improvements General principe 2 • A continuous evaluation approach • Enabled through the annual feature of the operation • After each campaign, decisions of evolutions • Implemented in N+1, but mostly N+2 or N+3

14. Year N-1 Year N Juin Jui. Août Sep. Oct. Nov. Déc. Jan Fév. Mar Avril Mai Juin Jui. Août Sep. Nov. Déc. Jan The schedule of a campaign Oct. RIL Expertise Sampling Setting-up of organisation Preparation Training Communication Collection Reception Controls Data processing Dissemination

15. Any improvement as effects at several points of the process : formulars, formations, protocols, softwares, traitements,… • Sometimes an improvement has his first impact within 2 or 3 years.

16. The national commission for the evaluation of the census • Under the rule of the CNIS (National Council for Statistical Information) • Presided over by a senator • It associates communes, the data users (researchers, administrations, municipalities), the Insee • It enables an exchange on data collection, control protocols, useful in improving the processes • It must give an opinion on the texts governing the organisation of the census

17. The management of quality, process by process : some exemples • Sampling frames • Data collection • Data processing • Dissemination

18. 1 - Sampling frames • The « quality » survey of the RIL (adress register in large communes) • An annual survey to assess • The surpluses and the deficits • The surplus-deficit balance • The communes for which the quality is deemed mediocre • The sample in 2008: 340 000 addresses and 900 000 dwellings • The results • 1% of dwellings in deficit, 1% in surplus • one commune out of eight beyond 2.5% of deficit • A level of quality higher than that of usual geographical registers • A regular improvement process

19. Sampling frames (2) • The action plans • To improve the quality of the RIL with targetted operations • Decided and implemented by the Regional Directorates in relation with the communes • As a function of the strong points and the weak points detected locally • Training of communes, fieldwork… • Allowing for an operational management of quality because targetted on specific problems and based on the closeness between the regional teams and the communes • The future efforts: reduce the surpluses and arrive at level surplus-deficit balance

20. 2 - Data collection • The control procedures during the data collection • The follow-up and control upon the initiative of the communes • The controls at the Insee after the data collection (reception-control, in office control, controls in the field)

21. The control procedure during data collection • A precise data collection protocol • Checked by the municipal officer • And by the supervisor

22. Follow-up and control system upon the initiative of the communes • Upstream from the data collection • Validation of the sample • Control-validation of the field to be observed carried-out by the enumerator • Follow-up of the data collection • Follow-up of the progress of work of the enumerator • Follow-up of key indicators : FLNE(Non response housing forms), vacant dwellings, quality of the log book of the tour-visits

23. The controls at the Insee after data collection • Successive « sorting » • The step of reception-registering of the questionnaires • Establishment of indicators • Communes with insufficient scores subject to in-depth control (approx. 15% of the communes representing 30% of the population) • Desk controls • Verification with the help of the housing Tax files • On the exhaustiveness of the dwellings, the FLNE, the structure of main/not main residence • In field controls

24. The in-field controls • They are done by Insee agents • They relate to the communes whose quality is presumed insufficient • Most often, they confirm the data collection • The errors detected are corrected • N.B. these controls are not representative, therefore their results cannot be extrapolated

25. Results of controls of FLNE • In 85% of the cases, the household enumerator finds the information • Most often, the FLNE does concern a main residence and the number of persons is correctly estimated • In 15 % of the cases, FLNEs have been wrongly provided (for non-main residences) • !!! These figures are not representatives. Controls are made on most difficult cases, where the proba of error is higher

26. 3.1 - Data processing : Data entry • outsourced to indepent service provider • Quality criteria specifically defined in the specifications • securing of transports and data • maximum rate of error by category of variables • … and evaluated by an independent control process (sample that has been input twice and arbitration)

27. 3.2 – Data procesing : coding of industry and profession • sample Quality control by twice coding : • Assessing the quality of the automatic coding and manual correction • Estimating a « threshold » of percentage of bulletins « non codable » • The contributions of this approach : • adding to the expert systemfiles • the improvement of training and protocols (precisions added in codification rules) • A finer management of quality, without going in for « excess quality »

28. 3.3 - Data processing - Data editing and imputation • Anayse the share of non responses or incoherent responses • Imputation of missing data or data presumed non-coherent (hot-deck procedure) • They are fine-tuned year after year, in view of the quality of the variables produced

29. 4 - Dissemination • A step prepared with the users • Define with them the products and services : two groups of the CNIS • Assess their satisfaction : groups of the CNIS and satisfaction surveys • A systematic validation of the data before dissemination • At the national level and in each regional directorate since 2009 • Today only at national level.

30. Any questions ?