1 / 42

Model Data Interoperability for the Integrated Ocean Observing System (IOOS)

Model Data Interoperability for the Integrated Ocean Observing System (IOOS). Rich Signell USGS Woods Hole/NOAA Silver Spring USA CENCOOS: July 14, 2009. Model Data Interoperability is Low-Hanging Fruit!. The Problem: Stovepiped Model Access.

moanna
Download Presentation

Model Data Interoperability for the Integrated Ocean Observing System (IOOS)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Model Data Interoperability for theIntegrated Ocean Observing System (IOOS) Rich Signell USGS Woods Hole/NOAA Silver Spring USA CENCOOS: July 14, 2009 Model Data Interoperability is Low-Hanging Fruit!

  2. The Problem: Stovepiped Model Access

  3. Result: Inability to compare and assess model results & underutilization of model products The GoMOOS Nowcast/Forecast Circulation Model (University of Maine)

  4. The Interoperability Experiment on Model Data in the Gulf of Maine • Gulf of Maine Ocean Data Partnership Modeling Committee: Five different groups (UMAINE, UMASS, BIO, WHOI and UNH) • Each group had: • A different oceanographic, met or wave model: POM, ECOM, ROMS, WaveWatch 3, WRF • A different toolkit for analysis • But each group also had: • NetCDF output files • A public web server (HTTP/1.1)

  5. A Simple NetCDF File

  6. NetCDF • Machine independent, self-describing, binary format for multidimensional scientific data • Efficient hyperslab extraction of data • nc_varget(file,var,corner,edge,stride) • nc_varget(‘etopo2.nc’,’topo’,[0 0],[20 30],[1 2]) • Unidata interfaces: Fortran, C, C++, Java • Third party interfaces: Perl, IDL, Python, Matlab • MIT-style license, supported by NSF at Unidata

  7. OPeNDAP (f.k.a. DODS) • Open-source Project for a Network Data Access Protocol: delivery of multidimensional scientific data via http • OPeNDAP allows efficient slicing from data via the web, just as NetCDF works for local files. • http://coast-enviro.er.usgs.gov/thredds/dodsC/ETOPO2v2?topo[0:19][0:29]

  8. Issue: Ocean grids are not regularly spaced! Curvilinear orthogonal horizontal coordinates Stretched surface and terrain following vertical coordinates

  9. NetCDF Climate and Forecast (CF) Conventions provide a solution Groups using CF: GO-ESSP: Global Organization for Earth System Science Portal IOOS: Integrated Ocean Observing System ESMF: Earth System Modeling Framework OGC: Open Geospatial Consortium (GALEON: WCS profile)

  10. CF Conventions for horizontal coordinates

  11. CF Conventions for stretched vertical coordinates

  12. CF-Compliant ROMS File

  13. NcML to the Rescue! (XML markup language for NetCDF)

  14. Data Interoperability Model

  15. NJ Toolkit for Matlabhttp://njtbx.sourceforge.net • Objective: Make it simple to access CF data • Example function: • [t, geo]=nj_tslice(URI,’temp’,1); • t = 22x120x180 single • geo = • lat: [120x180 single] • lon: [120x180 single] • z: [22x120x180 double] • time: 733582 (matlabdatenum) • nj_tslice works identically for ROMS, POM, ECOM, WRF, Wavewatch3 • URI can be: local NetCDF, remote NetCDF, NcML, OpenDAP Data URL • Mostly by Sachin Kumar Bhate MSSTATE

  16. THREDDS Data Server

  17. GOMODP THREDDS Catalog

  18. Comparing Models with Data in Matlab Model 1: UMASS-ECOM Model 2: UMAINE-POM Data: SST 2008-Sep-08 07:32

  19. Comparing Models with Data in Matlab

  20. Jim Manning’s drifter comparison

  21. 3D visualization of data with IDV

  22. Web Mapping Service with ncWMS

  23. Buoy_comp.m

  24. Demo

  25. Extracting Bathymetry Data:Step1: Browse http://blackburn.whoi.edu:8081/thredds for OpenDAP URL

  26. Extracting Bathy Data: Step 2: Use cf_subsetGrid.m with OpenDAP URL

  27. bathy_comp.m

  28. Using Mirone for Bathymetry

  29. Exported from Mirone to Iview3D

  30. Summary • Technology exists for standards-based delivery of aggregated geospatial data from native model grids that puts little or no effort on the data provider. It’s ready right now. • Demonstrated interoperability with 5 different models in the Gulf of Maine • What did we get? • Single point of access for regional model results • Interoperable software • More eyes on the model results • More usage of model results • Faster feedback to modelers • Improved models! • We can replicate this success in each IOOS region and with the National Backbone models

  31. NOAA IOOS 12 Month Work Plan (Dec 08-Nov 09) • Establish aggregated CF-compliant native-grid model data delivery by federal backbone providers (NAVY, NOAA) and IOOS regions • Facilitate development of: • CF Toolbox for Matlab • Unstructured grid class in NetCDF-Java • Converting existing tools to CF-compliant tools • Determine overlaps/collaboration opportunities with other groups with similar goals (e.g. OOI-CI, Unidata, BODC, IMOS)

  32. Questions, Feedback? Dr. Richard P. Signell+1 (508) 457-2229rsignell@usgs.govrich.signell@noaa.gov Try the Matlab CF Toolbox at: Gulf of Maine Modeling Pilot Project Page:http://www.gomodp.org/committees/modeling-committee/gulf-of-maine-model-interoperability-pilot-project Or just google “GOMODP” and look for the “page moved” URL!

  33. Glossary (1/6) • CF Conventions: NetCDF Climate and Forecast Metadata Conventionshttp://cf-pcmdi.llnl.gov/(Standards for identifying geospatial coordinates in NetCDF files andgrid relationships) • DMAC: Data Management And Communications subsystem http://dmac.ocean.us/ (One of three subsystems of IOOS, providing the infrastructure that will link the Observing and the Modeling and Applications subsystems) • ECOM: Estuarine, Coastal, and Ocean Model. http://www.hydroqual.com/ehst_env_hyd.html (3D ocean circulation model developed principally by Alan Blumberg) • ERDDAP: Environmental Research Division's Data Access Program http://coastwatch.pfeg.noaa.gov/erddap/index.html (A web service that aggregates data from diverse remote sources and offers a simple, consistent way to access the data)

  34. Glossary (2/6) • ESMF: Earth System Modeling Framework http://www.esmf.ucar.edu/ (Software for building and coupling weather, climate, and related models) • GO-ESSP: Global Organization for Earth System Science Portal http://go-essp.gfdl.noaa.gov/ (Collaboration designed to develop a new generation of software infrastructure that will provide distributed access to observed and simulated data from the climate and weather communities) • IDV: Integrated Data Viewer http://www.unidata.ucar.edu/software/idv/ (Java-based software for analyzing and visualizing geoscience data) • IOOS: Integrated Ocean Observing System http://ioos.noaa.gov/ (Multidisciplinary system designed to collect, deliver, and use ocean information) • IWGOO: Interagency Working Group on Ocean Observations

  35. Glossary (3/6) • MAST: Modeling and Analysis Steering Team http://www.ocean.us/Models (Guide the development of ocean and coastal modeling activities in support of the goals of IOOS) • NcWMS: Web Map Service http://www.resc.rdg.ac.uk/trac/ncWMS/ (ncWMS is a Web Map Service for geospatial data that are stored in CF-compliant NetCDF files. ) • NcML: NetCDF Markup Languagehttp://www.unidata.ucar.edu/software/netcdf/ncml/(Used to standardize and aggregate non-standard NetCDF files) • NetCDF-Java: NetCDF-Java Libraryhttp://www.unidata.ucar.edu/software/netcdf-java/(Library that takes advantage of CF conventions, NcML to facilitatebuilding standards-based clients)

  36. Glossary (4/6) • NSF: National Science Foundation http://www.nsf.gov/ (independent federal agency created by Congress in 1950 "to promote the progress of science; to advance the national health, prosperity, and welfare; to secure the national defense…“) • OpenDap: Open-source Project for a Network Data Access Protocolhttp://opendap.org(allows efficient hyperslabbing of geospatial data) • POM: Princeton Ocean Model http://www.aos.princeton.edu/WWWPUBLIC/htdocs.pom/ (Sigma coordinate, free surface ocean model with embedded turbulence and wave sub-models, and wet-dry capability) • RCOOS: Regional Coastal Ocean Observing System (Generic term for one of the 11 regional observing systems in IOOS)

  37. Glossary (5/6) • ROMS: Regional Ocean Modeling System www.myroms.org (A free-surface, terrain-following, primitive equations ocean model widely used by the scientific community for a diverse range of applications) • SCCOOS: Southern California Coastal Ocean Observing System http://www.sccoos.org/ (Brings together coastal observations in the Southern California Bight to provide information necessary to address issues in climate change, ecosystem preservation and management, coastal water quality, maritime operations, coastal hazards and national security. One of the 11 IOOS regions.) • THREDDS (Thematic Realtime Environmental Distributed Data Services) http://www.unidata.ucar.edu/projects/THREDDS (Services for delivering standards-based geospatial data via OpenDAP, WCS and more, works with NcML)

  38. Glossary (6/6) • WCS: Web Coverage Servicehttp://www.opengeospatial.org/standards/wcs(OGC standard for serving gridded information: time series, profiles,hyperslabs of remote sensing data, bathymetry, model output) • WW3: WaveWatch III http://polar.ncep.noaa.gov/waves/wavewatch/wavewatch.html (Ocean wave model developed by Hendrik Tolman) • WRF: Weather Research and Forecasting Model http://www.wrf-model.org/index.php (Next-generation mesocale numerical weather prediction system designed to serve both operational forecasting and atmospheric research needs)

More Related