Metadata for the S-DWH ‒ an overview

1. Metadata for the S-DWH ‒an overview Lars-Göran Lundell Statistics Sweden

2. Why do we need metadata? 42 DON’T PANIC

3. Metadata and the S-DWH • “Metadata is the DNA of the data warehouse” — Ralph Kimball • “Metadata for the data warehouse environment is one of the most important aspects” — Bill Inmon • “Statistical metadata systems play a fundamental role in statistical organizations” — METIS

4. The Metadata Framework General metadata considerations Definitions and terminology Collecting and using metadata Available standards Metadata in the S-DWH General requirements Adapting to the layered S-DWH Organising metadata Governing metadata Considering standards

5. Metadata Categories Formalised Free-form Reference Structural Active Passive

6. Metadata subsets • Statistical metadata • Process metadata • Quality metadata • Technical metadata • Authorisation metadata • Data models • Deliverable 1.1 Statistical Process Quality Technical Authorisation Data models

7. Metadata Functionalities General metadata considerations Definitions and terminology Collecting and using metadata Available standards Metadata in the S-DWH General requirements Adapting to the layered S-DWH Organising metadata Governing metadata Considering standards

8. Metadata Functionalities • Metadata functionalities needed to facilitate and support the operation of the S-DWH • Descriptions, definitions • Standards: GSBPM, Neuchâtel, etc. • Examples • Case study • Deliverable 1.4

9. Metadata Models General metadata considerations Definitions and terminology Collecting and using metadata Available standards Metadata in the S-DWH General requirements Adapting to the layered S-DWH Organising metadata Governing metadata Considering standards

10. Metadata Models • Overview of available models and standards • Relevant models for the S-DWH • Topicality, support, usage, usability • Suitability for metadata subsets • No super model • Recommendations • Keep it simple • Use only one model per subset • Make models/standards known • Deliverable 1.3

11. Metadata Quality General metadata considerations Definitions and terminology Collecting and using metadata Available standards Metadata in the S-DWH General requirements Adapting to the layered S-DWH Organising metadata Governing metadata Considering standards

12. Metadata Quality • International Standards • ISO 9000, ISO 11179 • Quality dimensions to assess S-DWH metadata Relevance Comparability Stability Accuracy Coherence Completeness Accessibility Uniqueness Interpretability • Metadata quality characteristics by S-DWH layer • Quality management for S-DWH metadata • Customer focus • Process approach • System approach to management

13. Metadata Quality • Recommendations: • Adopt quality dimensions • Decide on quality indicators and quality levels • Adopt naming standards • Decide on compulsory attributes • Set up governance and assessment rules • ... all “as appropriate” • Deliverable 1.2

14. Metadata for the layered architecture General metadata considerations Definitions and terminology Collecting and using metadata Available standards Metadata in the S-DWH General requirements Adapting to the layered S-DWH Organising metadata Governing metadata Considering standards

15. Metadata for the layered architecture • Mapping: • Functionalities of the SDWH metadata system • (deliverable 1.4) • Functional architecture of the SDWH • (deliverable 3.3) • Examples • Metadata subsets and functionalities by S-DWH layer • Metadata in the functional architecture data flows • Deliverable 1.6

16. The Metadata Layer General metadata considerations Definitions and terminology Collecting and using metadata Available standards Metadata in the S-DWH General requirements Adapting to the layered S-DWH Organising metadata Governing metadata Considering standards

17. The Metadata Layer • One logical metadata store • The user has one place to search • Matches all S-DWH layers • Several physical stores (possibly) • Deliverable 1.1 Free-form Formalised Reference Statistical Structural Process Active Quality Passive A metadata item The data store The metadata layer

18. Metadata Governance General metadata considerations Definitions and terminology Collecting and using metadata Available standards Metadata in the S-DWH General requirements Adapting to the layered S-DWH Organising metadata Governing metadata Considering standards

19. Metadata Governance • Governance and management • Roles and functionalities • Governance roles: Who does what? • Governance functions • Governance and metadata subsets • Deliverable 1.5

20. Deliverables 1.1 Metadata Framework Lars-Göran Lundell 1.2 Recommendations on the impact of Metadata Quality Colin Bowler, Michel Lindelauf, Jos Dressen 1.3 Overview of the use of Metadata Models Jos Dressen, Michel Lindelauf, Harry Goossens 1.4 Definitions of the Functionalities of a Metadata System Ennok, Lundell, Bowler, De Giorgi, Kulla 1.5 Governance of Metadata Management Viviana De Giorgi, Michel Lindelauf 1.6 Mapping deliverable 1.4 to the Ideal Architecture Maia Ennok

21. Metadata Framework General metadata considerations Definitions and terminology Collecting and using metadata Available standards Metadata in the S-DWH General requirements Adapting to the layered S-DWH Organising metadata Governing metadata Considering standards