SeaDataNet - Pan-European infrastructure for ocean and marine data management

1. SeaDataNet - Pan-European infrastructure for ocean and marine data management A key component of EMODNET By Dick M.A. Schaap – MARIS (SeaDataNet Technical Coordinator) Brussels, 4th May 2010, Expert Group on Marine Research Infrastructure Supported by the European Commission

2. Unified access to distributed data centres • SeaDataNet is a 5 year project, funded by EU via FP6, and has succeeded Sea-Search • SeaDataNet is setting up and operating an efficient Pan-European distributed infrastructure for managing marine and ocean data by connecting: • 40 National Oceanographic Data Centres (NODC’s), national oceanographic focal points, and ocean satellite data centres, in Europe • these Data Centres are mostly divisions of major national marine research institutes and based in 35 countries, surrounding the European seas • Data sets are made easily accessible for downloading through a unique portal, while the data sets are stored and managed at the individual Data Centres.

3. 49 Partners = 40 Data Centres + ICES + IOC-IODE + JRC + specialists in statistics, products and informatics

4. Interoperability Interoperability is the key to distributed data management system success. This is achieved in SeaDataNet via: • Adopting the ISO 19115 metadata standard for all metadata directories • Providing XML Validation Services to quality control the metadata maintenance • Using common and controlled vocabularies, including international content governance • Providing standard metadata entry tools • Using harmonised Data Transport Formats for data sets delivery • Using common quality control protocols and flag scale • Adopting of OGC standards for mapping and viewing services • Using SOAP Web Services in the SeaDataNet architecture • Organising training and instruction workshops for transfer of expertise and hands-on training

5. SeaDataNet portal www.seadatanet.org

6. SeaDataNet Discovery Services • EDMED - Data Sets • EDMERP – Research Projects • CSR – Cruise Summary Reports • EDIOS – Monitoring systems • EDMO – Marine Organisations • CDI – Common Data Index Common tools for entry and maintenance, including Controlled Vocabularies, and common user interfaces

7. SeaDataNet unified User Interfaces EDMERP CSR EDIOS EDMO

8. SeaDataNet from data discovery to data download to analysis

9. Access to Data – Common Data Index (CDI) • The Common Data Index (CDI) is a fine-grained index to individual data measurements (such as a CTD cast or moored instrument record). • CDI XML Schema is based upon the ISO 19115 standard • Common Data Index (CDI) system includes a Shopping and Transaction mechanism for direct online data access through file downloads • Use of harmonised Data Transport Formats for data sets delivery • Authorisation and Authentication Services, based upon a Central User Register and vocabulary of Data Access Restrictions

10. Data Policy and implementation • Metadata • free and open access, no registration required • each data centre is obliged to provide the meta-data in standardized format to populate the catalogue services • Data and products services • the general case is free and open access • however users must register once in order to get a personal login - password • Web form to provide necessary information • User agrees with “SeaDataNet User Licence” • After processing, login/password sent by email (email check) • Licence is part of the SeaDataNet Data Policy, that is intended to be fully compatible with the Directive of the European Parliament and of the Council on public access to environmental information, the INSPIRE Directive, IOC, ICES, WMO, GCOS, GEOSS and CLIVAR data principles. • SeaDataNet Data Policy is an overarching policy, respecting any local policy • Each user gets a SeaDataNet Role; while each CDI record has a ‘Data access restriction’ label. The combination of role and access label determines ‘access OK’, ‘access to be discussed’ or ‘access denied’

11. SeaDataNet CDI – Data retrieval and downloading Check Status In RSM Search Request Confirmed Include in Basket Results Ready at DC x Download Shopping list Data SDN format Submit + Authentication

12. Delivery Services – system set-up and process

13. Delivery Services – downloading

14. SeaDataNet CDI – Quick Search interface Using an intuitive approach which is easy for users

15. SeaDataNet CDI – Quick Search interface With Show / Hide Map option for display of search results

16. CDI – Data Coverage CDI V1 coverage at end April 2010: > 760.000 CDI entries from 25 countries and 33 data centres; connecting the remaining 10 countries and extending data coverage is well underway

17. SeaDataNet presentation and analysis tools • Ocean Data View (ODV) • General data analysis and visualization software • >10,000 registered users • ODV software upgraded to ODV4 for: • Extending ODV‘s graphical display capabilities and interactive controls for automatic and visual quality control and data quality flagging • Seamless connection to SeaDataNet output: SDN ODV4 data formats, quality flag scale • Integration of DIVA gridding software • Important tool for the SeaDataNet regional Data Products • DIVAsoftware (Data-Interpolating Variational Analysis) allows to spatially interpolate observations on a regular grid in an optimal way. The analysis is performed on a finite element grid allowing for a spatial variable resolution and a good representation of the coastline and isobaths.

18. SeaDataNet presentation and analysis tools

19. SeaDataNet Regional Products services Temperature and salinity climatologies for the Arctic seas, Atlantic Ocean,Baltic sea, Mediteranean sea and the Black Sea regions Statistical output in NetCDF files, that are visible via WMS

20. Cross sector involvement • SeaDataNet cooperates and provides services for many other European and international groups and projects, contributing and safeguarding good data management. Examples: • EuroGOOS, maintaining the European Directory of Ocean observing systems (EDIOS) and quality controlling and archiving long-term data series. See www.edios.org • POGO, collecting and providing information on ocean-going research vessels, and their operators, planned and completed cruises. See www.pogo-oceancruises.org • GMES Marine Core Services for streamlining the provision of long-term archives for optimising marine forecast services • Partners in the Humboldt project (GMES) contributing to a European Spatial Data Infrastructure • Services for the international oil & gas industry for metocean data (SIMORC). See www.simorc.org

21. SeaDataNet Expansion • SeaDataNet standards and tools are being adopted by new EU-funded projects, that will have a dedicated Portal, but also will be fully integrated in the SeaDataNet infrastructure: • Up-Grade Black Sea SCENE project, (2009 – 2011) involving 6 NODCs and 35 other data holding institutes from the 6 Black Sea countries, to provide metadata and data access and to strengthen their national NODC networks. EU-funding 3.4 Million Euro. • CASPINFO project, (2009 – 2011) involving 12 institutes and private industry from the Caspian Sea region, to provide metadata and data access and to build their capacity for data management and user services. EU-funding 0.8 Million Euro. • Geo-Seas project, (2009-2012) involving 24 geological and geophysical data centres from 16 European countries (EuroGeoSurveys), to provide metadata and data access. EU-funding 4.9 Million Euro. • EuroFleets project, (2009-2012) involving 24 research institutes and data centres from 17 European countries, to optimise the management of research vessels and to streamline the flow of data from the research cruises to the data centre infrastructure. EU-funding 7.2 Million Euro. • In each project several SeaDataNet NODCs are participating for transfer of knowledge & expertise, to secure interoperability and to learn from new developments

22. SeaDataNet and EMODNET • EU has adopted a new Marine Directive and Maritime Policy. • In its Blue Book the Commission proposed to take steps in 2008 towards an overarching European Marine Observation and Data Network (EMODNET) • In the accompanying Action Plan the Commission proposed to prepare by 2009 a Road map for EMODNET. • EU Taskforce is lead by DG-MARE and comprises also representatives of DG Research, DG Enterprise (GMES), DG Environment, and EEA. • The Roadmap was released in April 2009 as a Commission staff Working Document.

23. SeaDataNet and EMODNET • The Roadmap provides an overview of the main data and information service categories to be covered, some of their sources and uses, as well as examples of benefits and added value of better integration, and it clarifies how this initiative relates to other initiatives. • The Marine Strategy Framework Directive aims to achieve environmentally healthy marine waters by 2020. EMODNET will provide data for that purpose. • EuroGOOS and ESF Marine Board together prepared an EMODNET Vision document as input for the Roadmap. It states: ‘EMODNET will be a network of existing and developing European observation systems, linked by a data management structure covering all European coastal waters, shelf seas and surrounding ocean basins, accessible to everyone’.

24. SeaDataNet and EMODNET

25. SeaDataNet and EMODNET • EMODNET is also considered as a significant observation and monitoring data system for the part of the Water Information System for Europe (WISE) that is being developed for dealing with marine information (WISE-Marine) and supporting the data and indicator needs for the initial assessments required by member States in 2012 by the Marine Strategy Framework Directive.

26. SeaDataNet and EMODNET • To further the aims set out, the Commission launched preparatory actions. These aim at setting up portals to grant access to certain types of data over a number of maritime basins. • 3 year data pilots started in 2009, harvesting from DG Research developments: • Geology – undertaken by EuroGeoSurveys consortium • Chemistry – undertaken by SeaDataNet consortium • Biology – undertaken by EurOBIS – MARBEF consortium • Hydrography – undertaken by sub SeaDataNet consortium • Marine Habitats – undertaken by JNCC consortium • Additional funding (22.5 MEuro) arranged for more data pilots and extending and operating existing pilots in the coming years. • Overall objectives are to assemble fragmented and inaccessible marine data into interoperable, contiguous and publicly available data streams for complete maritime basins.

27. SeaDataNet and EMODNET • All pilots adopt SeaDataNet standards and interconnect to the SeaDataNet CDI Directory and Access services, adopting the SeaDataNet philosophy and strategy to open data resources: • Organisations, managing data for the focus regions, are invited to contribute their data for internal project use in the data products. • The public data products reference to the source data via the CDI metadata. This provides a shop window for data providers. • Data providers can regulate access to their data sets by indicating their ‘data access restrictions’ in the CDI metadata. • Users can follow the references to the data owners and request access to the data using the SeaDataNet shopping mechanism. SeaDataNet has been adopted as core element in EMODNET for the data management component and this way is enabled to upgrade its status from project based to sustained operational infrastructure.

28. SeaDataNet and EMODNET • EMODNET has the promise of a substantial and structural funding from 2014 onwards for the data management component and the observation systems. • It can be seen from the preparatory actions that this has accelerated the convergence between different data sectors (geology, hydrography, chemistry, ….). There is an increasing willingness to harmonise, to become interoperable and to enable sharing of data in sectors and between sectors. • One explanation might be the fact that organisations want to qualify for structural funding; another might be the INSPIRE Directive. By converging and cooperating both conditions can be fulfilled in a joint effort and more influential.

29. SeaDataNet and EMODNET • So far EMODNET has a focus on improved management and distribution of existing data. • Still to come is analysing and stimulating the interconnecting of observation systems and analyses which data are structurally missing and should be provided to support user applications in an optimal way. This will be done in synergy with GMES, EuroGOOS and EEA (in-situ coordination).