1 / 9

The Earth System Grid: Turning Climate Datasets into Community Resources

The Earth System Grid: Turning Climate Datasets into Community Resources. David E. Bernholdt, ORNL on behalf of the Earth System Grid team at Argonne National Laboratory Lawrence Berkeley National Laboratory Lawrence Livermore National Laboratory Los Alamos National Laboratory

jerzy
Download Presentation

The Earth System Grid: Turning Climate Datasets into Community Resources

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Earth System Grid: Turning Climate Datasets into Community Resources David E. Bernholdt, ORNL on behalf of the Earth System Grid team at Argonne National Laboratory Lawrence Berkeley National Laboratory Lawrence Livermore National Laboratory Los Alamos National Laboratory National Center for Atmospheric Research National Oceanic and Atmospheric Administration Oak Ridge National Laboratory University of Southern California www.earthsystemgrid.org

  2. The growing importance of climate simulation data Results from the Parallel Climate Model (PCM) depicting wind vectors, surface pressure, sea surface temperature, and sea ice concentration. Prepared from data published in the ESG using the FERRET analysis tool by Gary Strand, NCAR. • DOE invests broadly in climate change research: • Development of climate models • Climate change simulation • Model intercomparisons • Observational programs • Climate change research is increasingly data-intensive: • Analysis and intercomparison of simulation and observations from many sources • Data used by model developers, impacts analysts, policymakers 2 Bernholdt_ESG_SC07

  3. Earth System Grid objectives To support the infrastructural needs of the national and international climate community, ESG is providing crucial technology to securely access, monitor, catalog, transport, and distribute data in today’s grid computing environment. HPChardware running climate models ESG Portal ESGSites 3 Bernholdt_ESG_SC07

  4. ESG facts and figures CMIP3 (IPCC AR4) Daily Downloads (through 7/2/07) Worldwide ESG user base

  5. Climate data tools Metadata catalog NcML (metadata schema) OPenDAP-G (aggregation and subsetting) Data management Data Mover Lite Storage Resource Manager Globus toolkit Globus Security Infrastructure GridFTP Monitoring and Discovery Services Replica Location Service Security Access control MyProxy User registration NCAR Cache ORNLHPSS NERSC NCAR MSS RLS SRM RLS SRM SRM LANL Cache RLS Data Search User Registration Catalogs Browsing Access Control Climate Metadata Data Download Data Subsetting Data Publishing Usage Metrics SRM LLNL Cache Monitoring Services RLS Web Browser Web Browser Data User Data Provider DML ESG architecture and underlying technologies First Generation ESG Architecture search browse download publish DISK Cache MyProxy OPeNDAP-G SRM SRM ESG Web Portal RLS MSS, HPSS: Tertiarydata storage systems

  6. Evolving ESG to petascale ESG Data System Evolution 2006 Early 2009 2011 • Central database • Centralized curated data archive • Time aggregation • Distribution by file transport • No ESG responsibility for analysis • Shopping-cart-oriented web portal • Testbed data sharing • Federated metadata • Federated portals • Unified user interface • Selected server-side analysis • Location independence • Distributed aggregation • Manual data sharing • Manual publishing • Full data sharing (add to testbed…) • Synchronized federation • metadata, data • Full suite of server-sideanalysis • Model/observation integration • ESG embedded into desktop productivity tools • GIS integration • Model intercomparison metrics • User support, life cycle maintenance CSSM, IPCC,satellite, In situ biogeochemistry,ecosystems CCSMIPCC ESG Data Archive Terabytes Petabytes

  7. Distribution Online Data Deep Archives Distribution Online Data CPU ESG Node ESG Node ESG Gateway (CCES) ESG Node ESG Node Web Portal Interfaces Applications Data & Metadata Holdings ESG Node ESG Node ESG Node Architecture of thenext-generation ESG Second Generation ESG Architecture Remote Application Clients (CDAT, NCL, Ferret, GIS, Publishing, OPeNDAP, DML, Modeling, etc.) Browser Clients • Petascale data archives • Broader geographical distribution of archives • across the United States • around the world • Easy federation of sites • Increased flexibility and robustness Web Portals Local, Remote, and Web Services Interfaces Cross-Cutting Concerns (security, logging, monitoring) Workflow & Orchestration Applications Components (data transfer, data publishing, search, analysis, visualization, post-processing, computation) ESG Gateway (CCSM) Web Portal Interfaces Applications Data & Metadata Holdings ESG Gateway (IPCC) Web Portal Interfaces Applications Data & Metadata Holdings Federated ESG Deployment

  8. Climate Data Repository and ESG participant ESG participant The team and sponsors National Oceanic& AtmosphericAdministration/PMEL National Center forAtmospheric Research Argonne National Laboratory Lawrence BerkeleyNational Laboratory Lawrence LivermoreNational Laboratory/PCMDI Oak RidgeNational Laboratory USC InformationScience Institute Los Alamos National Laboratory

  9. For more information… ORNL booth at SC2007 • David Bernholdt Other booths at SC2007 • ANL/Global Grid Forum (Booth 551) Ann Chervenak • LBNL (351) Arie Shoshani, Alex Sim • NCAR (361) Don Middleton Internet • http://www.earthsystemgrid.org • Esg-manage@earthsystemgrid.org 9 Bernholdt_ESG_SC07

More Related