1 / 23

Better data quality through global data and metadata sharing

Better data quality through global data and metadata sharing. Agne Bikauskaite and Håkan Linden Eurostat. Outline. Context A data sharing model The necessary preconditions Implementing Eurostat's data sharing strategy Conclusions and outlook. Context. General objectives

thea
Download Presentation

Better data quality through global data and metadata sharing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Better data quality through global data and metadata sharing Agne Bikauskaite andHåkan Linden Eurostat

  2. Outline • Context • A data sharing model • The necessary preconditions • Implementing Eurostat's data sharing strategy • Conclusions and outlook

  3. Context General objectives • Reduce reporting burden on NSIs • More efficient use of resources in International Organisation (IO) • Ensure high quality and consistency of data of official statistics • Improve global data exchange and dissemination

  4. A data sharing model Eurostat Data Validation EUMember state EUMember state EUMember state EUMember state European statistics: From national to Eurostat

  5. A data sharing model IMF, UN, WB, ILO, BIS, other IOs U S E R S OECD Eurostat - ECB EU countries OECD countries(non-EU countries only) Other countries (non-OECD countries only) Eurostat as international hub for European statistics

  6. The necessary pre-conditions • Internationally agreed technical and statistical standards • Internationally agreed data structures • Maintenance agreements • Internationally agreed data validation • Streamlined data exchange processes

  7. Statistical Data and Metadata Exchange(SDMX) It consists of technical and statistical standards, guidelines, an IT service infrastructure and IT tools. SDMX provides • technical/statistical standards • new exchange modes (hubs) • clear rules and responsibilities SDMX ISO IS 17369

  8. SDMX describes the data and metadata exchange Provision Agreement Organisation scheme SDMX Registry maintainer Concept Schemes Code lists DSDs Concepts

  9. Describing the data exchange Who? When? Who? How? Where? What? What?

  10. Organisation 1 Organisation 2 Organisation 3 Content-Oriented guidelines Cross-domain concepts and code lists Statistical subject-matter domains Metadata common vocabulary Recommendations to harmonise implementations interoperability

  11. Implementing Eurostat's data sharing strategy Standardisation of structural metadata • Code lists describe dimensions in data tables, giving a meaning to the data. • Code lists are based on: • official statistical classifications such as NACE, NUTS, ISCO, etc. • The ESS and SDMX Content Oriented Guidelines • domain specific codifications • A standard code list is a code list already harmonised • Standard code lists should be used all along the statistical business process: data design, collection, aggregation, dissemination, exchange, archiving.

  12. Implementing Eurostat's data sharing strategyRecommendations for the SCL creation

  13. Implementing Eurostat's data sharing strategySDMX standards into ESS structural metadata In purpose to improve quality of the data comparability and clarity is needed: • To use identical SCLs in the ESS and in the SDMX • To transpose the SDMX guidelines into the ESS code lists • To adapt the ESS standard codes into the SDMX DSDs

  14. Implementing Eurostat's data sharing strategyOverview of the ESS SCLs • 504 ESS CLs • 194 ESS SCLs released in Ramon • 12 fully SDMX compliant • 110 SDMX compliant (except Generic codes)

  15. Implementing Eurostat's data sharing strategyStandardisation of Reference Metadata

  16. Implementing Eurostat's Reference metadata sharing strategy • WASTE (end of life vehicles, packaging, electronic waste) • WINE • FARM STRUCTURE • MIP STATISTICS • HICP/ Compliance monitoring • EHIS (Education, health and social protection) • R&D (CIS 2012) • Annual crops • PRAG • ESAW • AES (Education, Science and Culture) • LCI (Labour Cost Index) • INFOSOC (Information Society) • BUSINESS REGISTER • HICP • LFS-Q, LFS-A • EU-SILC • FATS • STS (Short Term Statistics) • WASTE • AEI (Pesticides) • EDUCAT • JVC (Job Vacancy Stats) • PRODCOM • EXTERNAL TRADE (3rd countries) • COSAEA • URBANREG • R&D • TOURISM • PERMANENT CROPS • CENSUS • HOUSING PRICES HPS • Over 30 Eurostat domains are in various phases of ESS Reference metadata standardisation. • This concerns about 35% of all eligible Eurostat processes.

  17. Implementing Eurostat's data sharing strategy The Eurostat established methodology

  18. Implementing Eurostat's data sharing strategyin ESS

  19. Implementing Eurostat's data sharing strategyDevelopment of the technical infrastructure Key components: • SDMX Registries • The Euro-SDMX Registry • The Global SDMX Registry • SDMX Reference Infrastructure (SDMX-RI)

  20. Implementing Eurostat's data sharing strategyWhat is the EuroSDMX Registry(SER)? • Eurostat's implementation of the SDMX Registry specifications as published by the SDMX initiative sdmx.org. • Based on SDMX 2.1 (as published on April 2011) Also capable of importing and exporting SDMX 2.0 artefacts. • Allows browsing, searching, editing and subscribing to artefacts. • Advanced access control mechanism for distributed maintenance of artefacts controlling also their visibility.

  21. Home page Access to the content of the Registryadvancedsearch Access to the content of the Registrytextsearch Access to the content of the Registry by type Most recent items

  22. Conclusions • International data co-operation improves the production of accurate, comparable and coherent statistics; • SDMX promotes an incremental movement toward the data and metadata sharing model; • The increasing use of SDMX based statistical standards improves the quality of the underlying statistical processes; • The SDMX technical standards pave the ways for simplified exchange and dissemination processes helping to improve also timeliness and accessibility; • Statistical integration needs to go hand-in-hand with technical integration and standardisation.

  23. Outlook • Much more global data and metadata sharing in the years to come; • Common data validation and processing procedures are required (from structural validation to content information validation); • Better metadata driven statistics production systems: the use of standards throughout the processes in combination with common metadata registries ; • Better harmonised international reference metadata frameworks and templates; • Broadening the scope of SDMX (versioning of codes, disabling of dimensions, other formats like CSV, flat files etc.); • Interoperability between information models (GSIM, SDMX, DDI etc.).

More Related