Census quality evaluation: Considerations from an international perspective

1. Joint UNECE/Eurostat Meeting on Population and Housing Censuses Geneva, 13-15 May 2008 Census quality evaluation: Considerations from an international perspective Bernard Baffour and Paolo Valente UNECE Statistical Division

2. Presentation Outline • Introduction • Quality definitions and dimensions • Assessment of dimensions of quality • Quality evaluation issues for different census methodologies • International perspective: What role for the International Organisations?

3. Definition of Quality and Census Quality • Quality is defined as the ‘fitness for use’ of data • Census quality translates into assessment of information produced by the census • Census quality characterized in terms of six dimensions: • Relevance, Accuracy, Timeliness, Accessibility, Interpretability, Coherence,

4. Dimensions of Quality • Relevance: concerns the user requirements • Accuracy: concerns the reliability and precision • Timeliness: concerns the time frame • Accessibility: concerns the dissemination strategy employed • Interpretability: concerns how easy to understand the data is • Coherence: concerns the conceptual integrity of the census data

5. Assessment of the Dimensions of Quality • Accuracy: quantification of the different errors Coverage, content, operational errors: • Coverage Assessment: Post-Enumeration Survey, Demographic Analysis, Comparison to Alternative Sources • Content Assessment: Edit and Imputation • Operational Assessment: Rigorous review of all census processes

6. Assessment of Coherence and Interpretability • Internal Coherence • Validation and Verification • Checks for inconsistencies, incongruities • External Coherence • Comparison to other data sources • Interpretability aided by comprehensive metadata provision • Concepts, Methodology, Coding

7. Assessment of Accessibility, Relevance, and Timeliness • Dissemination program – mix of free and tailor-made census products and services • Availability of (non-disclosive) microdata • Management of user requirements and expectations • Clear, realistic targets • User and public consultations and feedback

8. Quality Evaluation Issues for Different Census Methodologies • Census methodology adopted influences the assessment of census quality • The same quality dimension may be measured in different ways for different types of censuses • Important implications of census methodology on quality evaluation

9. Quality Evaluation Issues for Different Census Methodologies • Census methodologies: • Traditional census • Register-based census • Combination of registers + full enumeration • Combination of registers + sample data • Traditional enumeration with yearly updates of characteristics • Rolling census

10. Selected Quality Evaluation Issues: 1. Traditional Census • Relevance: • Content of the census questionnaire • Selection of topics: users needs vs. constraints • Post census consultations • Accuracy: • Coverage and content errors evaluated with PES, Demographic analysis, comparison with other sources • Timeliness: • Respect of planned calendar of output releases • May be difficult to achieve due to size of operations

11. Selected Quality Evaluation Issues: 2. Register-Based Census • Relevance: • No questionnaire • Users needs vs. data available from registers • Implications of using data collected for admin. uses • Consultation and feedback from users • Accuracy: • Assess coverage and quality of data in registers • Assess data integration: Matching rates • Consistency of values from different sources • Comparison with data from last traditional census • Timeliness: • Different updating schedules for different registers

12. Considerations from an International Perspective • Some countries have long tradition and adopt several evaluation methods • Others have limited experience and need to develop capacity • Challenges in evaluating quality of non-traditional censuses

13. Activities at international level • CES Census Recommendations • UNSD Principles and Recommendations • Eurostat: Work on census quality criteria • UNSD: Plans for Handbook on census evaluation/PES

14. What should International Organisations do? • Produce and disseminate guidelines, manuals, methodological material? • Promote methodological work? • Are standards needed on census quality evaluation?

15. Thank you!