Third plenary meeting of seadatanet
This presentation is the property of its rightful owner.
Sponsored Links
1 / 16

Third Plenary Meeting of SeaDataNet PowerPoint PPT Presentation


  • 44 Views
  • Uploaded on
  • Presentation posted in: General

Third Plenary Meeting of SeaDataNet. Bob Keeley Integrated Science Data Management, (ISDM) Canada March, 2009. Overview. General comments Some information about other activities Canada USA Observations on work reported at this meeting General observations Suggestions

Download Presentation

Third Plenary Meeting of SeaDataNet

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Third plenary meeting of seadatanet

Third Plenary Meetingof SeaDataNet

Bob Keeley

Integrated Science Data Management, (ISDM)

Canada

March, 2009


Overview

Overview

  • General comments

  • Some information about other activities

    • Canada

    • USA

  • Observations on work reported at this meeting

  • General observations

  • Suggestions

  • Now and after SDN

SeaDataNet

Mar 2009


General comments

General Comments

Advantage and disadvantage of a yearly snapshot of the project. I see progress but miss details.

Overall, as last year, I am very impressed with the progress and achievements.

You have done a lot to further the setting of standards and this is very important.

You have the base of a full distributed system that has largely defeated the duplicates problems that plague us.

SeaDataNet

Mar 2009


What is happening in canada

What is happening in Canada?

DFO is responsible for a much of the oceanographic data collected.

Most research data come from the Science Sector of DFO in which ISDM is sited.

4 years ago, Science reorganized so that data management gained a much higher profile and resulted in increased funding

We now have a national committee to coordinate activities and roughly 1 million CAD each year for projects (no guarantees into future).

Objective is to knit together a whole data system from 7 distinct (largely independent) pieces.

SeaDataNet

Mar 2009


Scope of projects

Scope of projects

Fisheries data from trawl surveys, ecosystem work, larval fish, hydroacoustics, … for lots of species.

Biological data including plankton measurements, marine mammal and other species tracking.

Lots of chemistry and physical oceanography

Data rescue projects from both marine and freshwater.

Digital images (videos, stills, microarray spectra, …)

Web services

Data catalogue and metadata (MCP)

Gridded bathymetric product served through web services

Use of taxonomy (ITIS)

Physical sample preservation and tracking.

SeaDataNet

Mar 2009


Data at isdm

Data at ISDM

Physical oceanography: delayed mode and lots received in real-time from GTS including from ships, profiling floats, surface drifters, wave buoys, tide gauges.

Biological: Invasive species, plankton

Chemical: contaminants, nutrients, etc.

Bathymetric: all hydrographic charts for Canada plus ENC delivery

SeaDataNet

Mar 2009


Data volumes

Data Volumes

Largest physical oceanography comes from surface drifters. Archive holds global measurements with > 75 million reports and growing at ~ 1 million/month

Global real-time profiles (Argo, ships, moorings, …): > 8 million stations and growing about 3 million/year (last year)

Wave spectra: ~ 10 million, adding 300,000/year

Tide gauges: ~ 4 million variable-days, adding 120,000/year

Bathymetry: lots and lots of individual soundings!

SeaDataNet

Mar 2009


Surface drifter map

Surface drifter Map

Was ~ 33 million in 2004, now is 75 million.

SeaDataNet

Mar 2009


Summary

Summary

If nothing else, we have large data volumes to deal with though the variety at ISDM is perhaps less than some of you have. Including the data at regional institutes, we have lots of variety.

We have the same issues about placing large archives on-line for download. Some of our requests are very large.

We have same issues of adopting/creating standards.

We need a robust architecture to deal with additional kinds of data.

We need to serve increasingly more varieties of data on-line.

SeaDataNet

Mar 2009


What is happening in u s

What is happening in U.S.?

DMAC activity in the US has been reinvigorated in last 18 months

They have also been pursuing standards some (many?) of which are different from yours

They are also are building a distributed, interoperable system without (I believe) an equivalent of CDI, CSR, etc.

SeaDataNet

Mar 2009


Observations on work reported at this meeting 1

Observations on work reported at this meeting - 1

There seems to have been some intercomparisons of QC at least with WODB. Has there been a check of internal consistency? I would like to see a document.

The CDI at present is roughly 1/2 million records. How will the performance scale when this is 10X, 100X larger? (Roy says no problem).

Some partners already deal in real-time / operational data and from a variety of sources. This starts to look like the real-time data we process. Does SDN want to open this door further?

It will be interesting to propose ODV/DIVA to ICOADS.

SeaDataNet

Mar 2009


Observations on work reported at this meeting 2

Observations on work reported at this meeting - 2

The download limitation of 500 records / transaction is a (near) future problem. Perhaps a solution is to allow larger single transactions but treat them as you do requests for data from centres without the download manager? (i.e. off-line, off peak processing)

Climatology generation needs careful validation. WOA is used extensively and so you should prove your results against that (and OI from CORIOLIS). Do sensitivity analyses of results to CL, S/N.

Andrew Wolf provided a favourable review of SDN. Did he comment on IT security issues? Can others use his comments to assure their own IT staff of precautions taken to secure the system? A document outlining these precautions would be nice.

SeaDataNet

Mar 2009


Observations from this meeting

Observations from this meeting

  • Up until now, SDN has (rightly) focused inwards on getting your project running. I see that though not complete, you have a base running.

  • I would suggest it is time to begin to look outward for a couple of reasons:

    • Your standards developments are important to more than yourselves.

    • As SDN completes, this is a way to grow what has been produced and sustain the initiative into the future.

SeaDataNet

Mar 2009


Suggestions to consider

Suggestions to Consider

DMAC and SDN are diverging. Send someone to the next possible DMAC meeting to do a presentation.

Share your standards work with the world. Use the OceanDataStandards web site and process to do this. (see http://oceandatastandards.org )

JCOMM and IODE are developing the ODP in support of a funded pilot project from WMO. But ODP and SDN seem to be on different development paths. What can be done to remedy this? Do you want 2 systems in Europe?

SeaDataNet

Mar 2009


Now and after sdn 1

Now and after SDN - 1

SDN provides the funding to build a distributed system, but the “glue” that keeps these together is the coordination activities. After SDN, without this coordination, the system risks breaking.

Use the international systems, IODE, JCOMM, to help continue the coordination into the future even if you secure more EU funding.

When funding ends, what will other partners do if one drops out?

OceanObs’09 will take place in September this year. A plenary paper will set out the 10 year objective for ocean data management. You want to have a voice.

SeaDataNet

Mar 2009


Now and after sdn 2

Now and after SDN - 2

The WDC system is undergoing a large change right now. Given that SDN is a distributed system that preserves data in the long term, what do you want from the WDCs?

What is the relationship between SDN and the various DACs that now exist?

What connections would you want to make to the various data collection panels that operate under JCOMM?

What do you want from and what do you want to contribute to IODE and JCOMM?

SeaDataNet

Mar 2009


  • Login