UNECE-Eurostat-OECD Meeting on Management of Statistical Information Systems, MSIS 2008
Download
1 / 16

Luxembourg, 07 - PowerPoint PPT Presentation


  • 82 Views
  • Uploaded on

UNECE-Eurostat-OECD Meeting on Management of Statistical Information Systems, MSIS 2008 . Integrate architecture using SDMX - A framework for SDMX. Laura Vignola National Statistical Institute – Italy (Istat). Luxembourg, 07/04/2008.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Luxembourg, 07' - madelyn


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Slide1 l.jpg

UNECE-Eurostat-OECD Meeting on Management of Statistical Information Systems, MSIS 2008

Integrate architecture using SDMX - A framework for SDMX

Laura Vignola

National Statistical Institute – Italy (Istat)

Luxembourg, 07/04/2008


Slide2 l.jpg

UNECE-Eurostat-OECD Meeting on Management of Statistical Information Systems,MSIS 2008

UNECE-Eurostat-OECD Meeting on Management of Statistical Information Systems,MSIS 2008

Istat situation of data dissemination

ISTAT situation of data dissemination

A lot of sub - system web with data stored in different db server and with different structure

Tables excel, Gesmes files, other formats

Luxembourg, 07-09/04/2008


Slide3 l.jpg

UNECE-Eurostat-OECD Meeting on Management of Statistical Information Systems,MSIS 2008

First phase of exchange data in SDMX format

Short term indicators database - CONISTAT

1) Mapping between metadata alredy present in CONISTAT and the DSD done by Eurostat

2) Realizing a WS who accept an SDMXquery and retrive an SDMX compact

Luxembourg, 07-09/04/2008


Slide4 l.jpg

UNECE-Eurostat-OECD Meeting on Management of Statistical Information Systems,MSIS 2008

Second phase of exchange data in SDMX format

Solutions:

Long terms - to implement a registry/hub, will let end - user to access all data, now distribuited in different database and data warehouse, from a single entry point

Short terms - to implement a registry/repository accessible from a single web service. A set of re-usable modules that drive the data flow from the reporting to dissemination

Luxembourg, 07-09/04/2008


Slide5 l.jpg

UNECE-Eurostat-OECD Meeting on Management of Statistical Information Systems,MSIS 2008

Second phase of exchange data in SDMX format

A short term solution – a framework for SDMX

.ges

.txt

ESA

Rss

EUROSTAT

DEM

STS

SDMX query

LOADER

Query parser

SDMX Data gen.

SDMX DB

SDMX data file

WEB SERVICES

Luxembourg, 07-09/04/2008


Slide6 l.jpg

UNECE-Eurostat-OECD Meeting on Management of Statistical Information Systems,MSIS 2008

Structure of database

DESCRIPTORS

COMPONENT_DESCRIPTOR

KEYFAMILIES

PK_DESCRIPTOR

FK_KEY_FAMILY

DESCRIPTOR_TYPE

GROUP_ID

KEYFAMILY

NAME

AGENCY

VERSION

ISFINAL

FLAG_CS

FK_DESCRIOPTOR

FK_KEYFAMILY

FK_COMPONENT

COMPONENT_DESCRIPTOR_TYPE

DATA_FLOWS

PK_DATAFLOW

DESCRIZIONE

FK_KEY_FAMILY

COMPONENTS

CROSSSECTIONAL_AL

PK_COMPONENT

FK_KEYFAMILY

FK_CODE_LIST

COMPONENT_TYPE

ORDERC

USAGE_STATUS

ATTACHMENT_LEVEL

USED

PK_COMPONENT

FK_KEY_FAMILY

CROSSSECTIONALATTACHDATASET

CROSSSECTIONALATTACHGROUP

CROSSSECTIONALATTACHSERIES

CROSSSECTIONALATTACHOBS

CODE_LISTS

PK_CODE_LIST

FK_KEY_FAMILY

NAME

AGENCY

CODES

PK_CODE

FK_KEY_FAMILY

FK_CODE_LISTS

DESCRIPTION

METADATI_STS

METADATI_DEMOGRAPHY

SSTSIND_PROD_M

SSTSIND_ORD_M

FREQ

REF_AREA

ADJUSTMENT

STS_INDICATOR

STS_ACTIVITY

STS_INSTITUTION

STS_BASE_YEAR

TIME_FORMAT

FK_DATAFLOW

DEMOGRAPHY1

FREQ

COUNTRY

SEX

DEMO

TAB_NUM

REV_NUM

UNIT_MULT

DECI

TIME_FORMAT

UNIT

FK_DATAFLOW

FREQ

REF_AREA

ADJUSTMENT

STS_INDICATOR

STS_ACTIVITY

STS_INSTITUTION

STS_BASE_YEAR

TIME_PERIOD

OBS_VALUE

OBS_STATUS

OBS_COM

OBS_CONF

FREQ

REF_AREA

ADJUSTMENT

STS_INDICATOR

STS_ACTIVITY

STS_INSTITUTION

STS_BASE_YEAR

TIME_PERIOD

OBS_VALUE

OBS_STATUS

OBS_COM

OBS_CONF

OBS_PRE_BREAK

FREQ

COUNTRY

SEX

DEMO

TIME_PERIOD

OBS_VALUE

OBS_STATUS

Luxembourg, 07-09/04/2008


Slide7 l.jpg

UNECE-Eurostat-OECD Meeting on Management of Statistical Information Systems,MSIS 2008

Client Server Application

Luxembourg, 07-09/04/2008


Slide8 l.jpg

UNECE-Eurostat-OECD Meeting on Management of Statistical Information Systems,MSIS 2008

Client Server Application

Luxembourg, 07-09/04/2008


Slide9 l.jpg

UNECE-Eurostat-OECD Meeting on Management of Statistical Information Systems,MSIS 2008

Client Server Application

Luxembourg, 07-09/04/2008


Slide10 l.jpg

UNECE-Eurostat-OECD Meeting on Management of Statistical Information Systems,MSIS 2008

Client Server Application

Luxembourg, 07-09/04/2008


Slide11 l.jpg

UNECE-Eurostat-OECD Meeting on Management of Statistical Information Systems,MSIS 2008

Client Server Application

Luxembourg, 07-09/04/2008


Slide12 l.jpg

UNECE-Eurostat-OECD Meeting on Management of Statistical Information Systems,MSIS 2008

SDMX Data Where Query

  • <Query>

  • <query:DataWhere>

  • <query:And>

  • <query:DataFlow>STSIND_PROD_M</query:DataFlow>

  • <query:Dimension name="FREQUENCY">M</query:Dimension>

  • <query:Dimension name="INDICATOR">PROD</query:Dimension>

  • <query:Or>

  • <query:And>

  • <query:Dimension name="ADJUSTMENT">S</query:Dimension>

  • <query:Or>

  • <query:Dimension name="STS_ACTIVITY">N100DA</query:Dimension>

  • <query:Dimension name="STS_ACTIVITY">N100C0</query:Dimension>

  • <query:Dimension name="STS_ACTIVITY">N100D0</query:Dimension>

  • </query:Or>

  • <query:Time>

  • <query:StartTime>2000-01</query:StartTime>

  • <query:EndTime>2006-01</query:EndTime>

  • </query:Time>

  • </query:And>

    • <query:And>

    • <query:Dimension name="STS_ACTIVITY">N100C0</query:Dimension>

    • <query:Time>

    • <query:StartTime>1995-01</query:StartTime>

    • <query:EndTime>2003-01</query:EndTime>

    • </query:Time>

    • </query:And>

  • </query:Or>

  • </query:And>

  • </query:DataWhere>

  • </Query>

Luxembourg, 07-09/04/2008


Slide13 l.jpg

UNECE-Eurostat-OECD Meeting on Management of Statistical Information Systems,MSIS 2008

The query parser

Db independent side of web services - rewriting the xml file

<Time>

<StartTime>Date 1</StartTime>

<EndTime>Date 2</EndTime>

</Time>

<And>

<StartTime>Date 1</StartTime>

<EndTime>Date 2</EndTime>

</And>

<And>

<And> Cond 1, Cond 2 </And>

</And>

<And>

Cond 1

Cond 2

</And>

Luxembourg, 07-09/04/2008


Slide14 l.jpg

UNECE-Eurostat-OECD Meeting on Management of Statistical Information Systems,MSIS 2008

The query parser

<Or>

<Or> Cond 4, Cond 5 </Or>

</Or>

<Or>

Cond 1

Cond 2

</Or>

<And>

Cond 1

<Or> Cond 2, Cond 3</Or>

</And>

<Or>

<And> Cond 1, Cond 2</And>

<And> Cond 1, Cond 3 </And>

</Or>

Luxembourg, 07-09/04/2008


Slide15 l.jpg

UNECE-Eurostat-OECD Meeting on Management of Statistical Information Systems,MSIS 2008

SDMX Data file generator

Db dependent side of web services

SELECT METADATI_STS.*,SSTSIND_PROD.* FROM METADATI_STS,SSTSIND_PROD WHERE ADJUSTMENT=‘S’ AND FREQUENCY=‘M’ AND INDICATOR=‘PROD’ AND STS_ACTIVITY=N100DA AND TIME_PERIOD BETWEEN 2000-01 AND 2006-01

UNION

SELECT METADATI_STS.*,SSTSIND_PROD.* FROM METADATI_STS,SSTSIND_PROD WHERE ADJUSTMENT=‘S’ AND FREQUENCY=‘M’ AND INDICATOR=‘PROD’ AND STS_ACTIVITY=N100D0 AND TIME_PERIOD BETWEEN 2000-01 AND 2006-01

………………………………………………

Luxembourg, 07-09/04/2008


Slide16 l.jpg

UNECE-Eurostat-OECD Meeting on Management of Statistical Information Systems, MSIS 2008

Future developement

- Introduce new features on the existing modules

- Make available some of this modules as API

Luxembourg, 07/04/2008


ad