1 / 13

WGISS-47 NCEI Data Access

Discover how the WGISS-47 NCEI Data Access Committee promotes interoperable capabilities for Earth observation resources, facilitating data discovery, access, and utilization for the international community of science and applications users.

whiteedward
Download Presentation

WGISS-47 NCEI Data Access

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. WGISS-47 NCEI Data Access Committee on Earth Observation Satellites Rich Baldwin, Chief NOAA/NESDIS/NCEI Data Access Branch CEOS WGISS-47 Silver Spring, MD, USA 29 April 2019

  2. NOAA Connections to WGISS “The Forum” • The primary benefit for NOAA is that WGISS provides a forum for space agency information system developers to exchange ideas and approaches as they implement interoperable capabilities that promote and enable the discovery, access and utilization of their Earth observation resources by the broad international community of science and applications users • With the constant evolution of data system technologies and geospatial standards the WGISS members can share their experiences and identify best practices in applying these capabilities • When and where appropriate, WGISS promotes collaboration in the development of interoperable infrastructure components such as the IDN, FedEO and CWIC.

  3. Data Access - Core Objectives • Build, manage and maintain systems which provide public access to archived data and metadata for human and machine discovery and use. • OneStop • Mission Data Management System • Common Access • THREDDS, ERDDAP, LAS • Enable data provisioning through the NOAA Big Data Partners • AWS, Google, Azure, OCC • Support and enable interoperable systems • CWIC • ESRI Geoportal

  4. CWIC - Interoperability OneStop Geoportal

  5. CWIC - Interoperability

  6. Supporting CWIC • NOAA re-organizing support for CWIC • OPPA to NCEI • Science Data Services contract • subcontract being arranged/negotiated • Support includes ... • Monitor GEOSS activity and communicate with the GEOSS GCI team to ensure CWIC data resources can be accessed by GEOSS. • Facilitate collaboration with the CEOS WGISS CWIC task team. • Contractor shall lead the project and will be responsible for architecture design, technical leadership, daily project team management, and CWIC promotion.

  7. Supporting CWIC • Support includes ... • The implementation of a science discipline data portal to facilitate the discoverability and accessibility of Essential Climate Variables (ECVs) products and space-borne Climate Data Records (CDRs). • Integrate more data centers into CWIC and enhance CWIC functionality. In addition, the Contractor shall work with GEO to integrate CWIC with GEOSS and their data systems. This task shall include detailed metric reporting, updates to CWIC documentation, and periodic progress reports. This task shall include but is not limited to: • Operating, updating, and maintaining CWIC operation

  8. NOAA’s Diversity of Data • NOAA collects, maintains, analyzes, produces, and serves a tremendous variety of data • From the surface of the sun to the bottom of the sea, and everything in between • From ships and satellites and radar and divers in the water and drifting buoys and autonomous platforms and … • Over 65,000 collections in our catalog with billions of data granules, provided by thousands of organizations, spanning centuries and the globe...

  9. WGISS Interoperable Infrastructure • By working together and leveraging each other's efforts and experience, WGISS members can help to ensure the maximum return on the collective investments of the CEOS agency Earth observation missions. • In NOAA’s case, Over 65,000 collections in our catalog with billions of data granules, provided by thousands of organizations, spanning centuries and the globe...

  10. Challenges What are some of the significant challenges that prevent growth and collaboration? • Funding and the timing associated with those fund (Government Support Services Contracts, Cooperative Research Institute, etc.) • Available time and attention • “Good Neighbor” Data Access • Awareness of limited system capacity - User DOS Attack • Greater use of subscriptions • Greater use of cloud data stores • Volume of Data • Increasing demand • Increase in need for specificity • Volume of Metadata • ElasticSearch clusters • 10+ million artifacts -> 100 fold growth per annum • Transitioning Users to Cloud • THREDDS - filesystem to object store • Partial https using NetCDF API and programs such as NASA’s Panoply

  11. Backup Slides

  12. Discovery and Access WGISS Connected Data Assets (CDA) Single entry point for external clients to discover/access CEOS agencies data Search over 32,000 collections in the IDN (collection search) Access over 5000 collections with associated over 300+ million granules (granule search) Borrowed from Mirko’s SIT-34 WGISS Presentation CEOS Service DATA-2: Full representation and accessibility of CEOS Agency datasets through WGISS Standards and Connected Data Assets Infrastructure (i.e. IDN, CWIC, FEDEO). DATA-9: ECVs/CDRs Discovery and Access through WGISS Systems. FDA-14:Facilitate discovery and access for end users to data analytics and processing tools and services through the WGISS Connected Data Assets Infrastructure.

  13. Support to CWIC A Critical Component of the SST-VC/GHRSST access framework is the CEOS-WGISS Integrated Catalog (CWIC) • Since WGISS/CWIC made excellent choices regarding the use of interoperable data discovery standards, and since the SST-VC and GHRSST utilized those same services, integration of the GHRSST R/GTS framework and CWIC has been a very straightforward process • GHRSST data collections are therefore fully discoverable through the CEOS International Directory Network (IDN) and data granules are fully discoverable through CWIC. SST-VC

More Related