slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
DMAC The Data Management and Communications Subsystem of the U.S. Integrated Ocean Observing System (IOOS) PowerPoint Presentation
Download Presentation
DMAC The Data Management and Communications Subsystem of the U.S. Integrated Ocean Observing System (IOOS)

Loading in 2 Seconds...

play fullscreen
1 / 25

DMAC The Data Management and Communications Subsystem of the U.S. Integrated Ocean Observing System (IOOS) - PowerPoint PPT Presentation


  • 127 Views
  • Uploaded on

DMAC The Data Management and Communications Subsystem of the U.S. Integrated Ocean Observing System (IOOS). Steve Hankin NOAA/Pacific Marine Environmental Laboratory DMAC Steering Team DMAC Modelers’ Caucus. GOOS - Global Ocean Observations GTSPP, ARGO, Satellites, VOS, others.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'DMAC The Data Management and Communications Subsystem of the U.S. Integrated Ocean Observing System (IOOS)' - wood


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

DMACThe Data Management and Communications Subsystemof theU.S. Integrated Ocean Observing System (IOOS)

Steve Hankin

NOAA/Pacific Marine Environmental Laboratory DMAC Steering TeamDMACModelers’ Caucus

slide2

GOOS - Global Ocean ObservationsGTSPP, ARGO, Satellites, VOS, others

Why not do something comparablefor the coastal ocean?

TECO-WIS

us ioos coastal system

Regional Associations

US IOOS Coastal System:

“National Backbone”

TECO-WIS

slide4

DMAC

Subsystem

Modeling & Analysis

Subsystem

Societal Goals

Safe & Efficient

Maritime Operations

Metadata standards

Data discovery

Data transport

Online browse

Data archival

Homeland Security

Natural Hazards

Climate Change

Public Health

Ecosystem Health

Living Marine

Resources

Integrated Ocean Observing System

Observing

Subsystem

Satellites

Aircraft

Fixed Platforms

Ships

Drifters & Floats

AUVs

TECO-WIS

many variables the ioos core set
Many Variables – the IOOS Core Set
  • Physical
    • Salinity, temperature, air-sea heat flux
    • Sea and water level, water depth (Shore line)
    • Surface waves (height, direction, freq), currents & sea ice
  • Chemical
    • Contaminants & dissolved nutrients
    • Dissolved oxygen
  • Biological
    • Fish species & abundance
    • Chlorophyll, ocean color & phytoplankton abundance/species
    • Zooplankton abundance/species
    • Pathogens
  • Interdisciplinary
    • Optical properties (Atmospheric & in-water)
    • Bottom character and benthic habitats

TECO-WIS

the governance challenge
The Governance Challenge
  • Many independent observing system activities
    • federal, state, county, municipal, academic & private players
  • Independent funding paths
  • Widely varying program missions
  • A planning office -- Ocean.US -- established to coordinate the efforts

Effective governance depends upon a workable data management and communications (DMAC) plan

TECO-WIS

slide7

…competingcharacteristics ruled out the most straightforward solutions for IOOS

TECO-WIS

slide9

No management structure exists to enforce adoption of a complex, data management solution

TECO-WIS

the dmac plan a community effort
The DMAC Plan – a community effort

Steering Comm. convened June 2002

  • Brought together …
    • 6 Federal agencies
    • 6 Universities
    • 4 Regional/State agencies
    • 3 Private sector orgs.
  • Steering Committee guided2 Outreach Teams,4 Expert Teams
    • Data Facilities Outreach
    • User Outreach
    • Data Discovery & Metadata
    • Data Transport
    • Data Archive & Access
    • Applications & Products

Public drafts circulated Dec. 2003

TECO-WIS

the dmac plan a community effort1
The DMAC Plan – a community effort
  • Four levels of review
    • Solicited expert review
    • Public workshop
    • Public email-based review
    • Federal Registry comments
  • Drafts broadly circulated
  • ExCom, NORLC, IWG, USGST, Ocean.US Workshops, Regional Summits, Professional Society newsletters, email lists and websites
  • 150 reviewers
    • 6 Federal Agencies
    • 22 Universities & Institutes
    • 13 Private Sector
    • 13 Regional/State agencies
    • 5 International

Published March 2005

  • An implementation plan(a roadmap to detailed specification)
slide13
The DMAC Plan is a framework for interoperability among independent, heterogeneous systems, large and small
  • Independence: Organizations will manage their own data
  • Standards: enabling innovation!
  • “Web services”: the common language to be spoken by pre-existing systems

TECO-WIS

how dmac selects standards
How DMAC selects standards

“Adapt, adoptand only as a last resortdevelop”

Strong preference for open standards

Classified by IOOS maturity levels

  • R&D
  • Pilot
  • Pre-operational
  • operational

TECO-WIS

dmac planning process continues
DMAC planning process continues …

Steering Team

Expert Teams

Transport and Access

Metadata and Data Discovery

Archive

Caucuses

Private sector

International

Modeling

Regional

Working Group

Systems Engineering

Interagency Oversight Working Group

TECO-WIS

data transport
Data Transport

“Adapt, adoptand only as a last resortdevelop”

  • Expert Team -- 10 members
    • academia (4), federal (4), private sector (2)

Harmonize developments from 2 communities

  • GIS – utilizing OGC W*F standards
  • FES* – utilizing OPeNDAP

* FES = fluid earth systems

TECO-WIS

digression on opendap dap data access protocol
Digression on OPeNDAP …”DAP” == Data Access Protocol

Since 1992 - a “web service” before the term existed

  • “Files” vanish  “datasets”, instead
  • On the server
    • netCDF, GRIB, HDF, IEEE, … or relational databases
    • aggregations (multi-terabyte datasets)
  • On the client
    • “open” and read subsets
    • any netCDF application is a web client (may need to relink)
    • Matlab, IDL, IDV, GrADS, Ferret, CDAT, …
  • Widely used in research modeling communities

TECO-WIS

data transport1
Data Transport

IOOS maturity levels

  • R&D
  • Pilot
  • Pre-operational
  • operational

TECO-WIS

metadata and data discovery
Metadata andData Discovery

“Adapt, adoptand only as a last resortdevelop”

Team approach

  • Coordination with organizations/programs such as MMI*, ISO, FGDC*, GCMD*, ORION* and QARTOD*
  • Engage data providers in the process

Determine marine metadata content requirements

  • data lifecycle: identify, assess, access, utilize and archive
  • adaptive to data type and method of data collection

Compare metadata elements across multiple standards

  • FGDC, ISO, DIF, EML, ESML, SensorML, MarineXML, etc.
  • MMI = Marine Metadata Interoperability project
  • FGDC = US Federal Geographic Data Committee
  • GCMD = NASA Global Change Master Directory
  • ORION = US NSF-funded ocean research initiative
  • QARTOD = Quality Assurance Real Time Ocean Data

TECO-WIS

metadata and data discovery1
Metadata andData Discovery

Starting point: “Guide for IOOS Data Providers, v1.0”

Initial guidance contains:

  • Required vocabularies
    • IOOS keywords, core variables, national backbone programs
  • Recommended vocabularies
    • ISO 19115 topic categories, GCMD keywords, IHO codes, OBIS species, CF standard names
  • Metadata standards
    • ISO 19115, FGDC CSDGM, GCMD DIF, OBIS
  • Minimal list of elements
    • Discovery, access, data transport, archive, consumer use, data management
  • Data discovery systems
    • NSDI Clearinghouse, Geospatial OneStop, GCMD, OBIS

TECO-WIS

slide21

DMAC Modelers’ Caucus – Challenges

  • Advancement and use of CF (netCDF & OPeNDAP)Further work:
    • unstructured grids
    • Geospatial datums
    • Nested grids
  • Downscaling and splicing of neighboring domains
  • Rich model metadata
  • OSSEs – couple modeling and observing system design
  • Model-data validation and model-model comparison

Modeling and Analysis Steering Team (MAST)

>> Convened Oct. 25, 2006 <<

Community Modeling Networks (CMNs) to follow

community standards for unstructured grids ucar unidata boulder co october 16 18 noaa ioos funded
“Community Standards forUnstructured Grids”UCAR/Unidata, Boulder CO -- >> October 16-18 <<NOAA-IOOS funded
  • Standards for unstructured coastal model outputs
    • A community process
    • Standardize file contents, formats
    • Standardize network access protocols
  • A demonstration testbed will be created
  • Technical discussions will be open
    • on a moderated web site

http://cf-pcmdi.llnl.gov/

TECO-WIS

dmac and wis closely related challenges
DMAC and WIS – Closely related challenges

Both seek to

  • Link organizationally-independent data suppliers
  • Address a broad range of user needs
  • Integrate heterogeneous data types
  • Merge real-time (push) and delayed mode (pull)
  • Build upon standards

WORLD METEOROLOGICALORGANIZATIONWeather  Climate  Water

TECO-WIS

dmac and wis closely coupled solutions
DMAC and WIS – Closely coupled solutions

DMAC areas in need of particular attention(to what degree can WIS solutions be adopted?)

  • Real time (push) ( IDD/LDM? )
  • Access control
  • Service registries
  • Data assembly (regional and national)

WORLD METEOROLOGICALORGANIZATIONWeather  Climate  Water

TECO-WIS