1 / 9

The Earth System Grid: Turning Climate Datasets into Community Resources

The Earth System Grid: Turning Climate Datasets into Community Resources. www.earthsystemgrid.org. The growing importance of climate simulation data.

yama
Download Presentation

The Earth System Grid: Turning Climate Datasets into Community Resources

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Earth System Grid: Turning Climate Datasets into Community Resources www.earthsystemgrid.org

  2. The growing importance of climate simulation data Results from the Parallel Climate Model (PCM) depicting wind vectors, surface pressure, sea surface temperature, and sea ice concentration. Prepared from data published in the ESG using the FERRET analysis tool by Gary Strand, NCAR. • DOE invests broadly in climate change research • Development of climate models • Climate change simulation • Model intercomparisons • Observational programs • Climate change research is increasingly data intensive • Analysis and intercomparison of simulation and observations from many sources • Data used by model developers, impacts analysts, policymakers 2 Bernholdt_0611

  3. Earth System Grid objectives To support the infrastructural needs of the national and international climate community, ESG is providing crucial technology to securely access, monitor, catalog, transport, and distribute data in today’s Grid computing environment. HPChardware running climate models ESG Portal ESGSites 3 Bernholdt_ESG_0611

  4. IPCC Downloads (10/12/06) Nov 2004 – Oct 2006 Worldwide ESG user base ESG facts and figures

  5. Climate data Metadata catalog NcML (metadata schema) OPenDAP-G (aggregation and subsetting) Data management Data Mover Lite Storage Resource Manager Globus toolkit Globus Security Infrastructure GridFTP Monitoring and Discovery Services Replica Location Service Security Access control MyProxy User registration ORNLHPSS NERSC NCAR MSS NCAR Cache RLS SRM RLS SRM DISK Cache MyProxy OPeNDAP-G SRM SRM ESG Web Portal LANL Cache Data Search User Registration Catalogs Browsing RLS RLS Access Control Climate Metadata Data Download Data Subsetting Data Publishing Usage Metrics Monitoring Services search browse download Web Browser Web Browser publish Data User Data Provider DML ESG architecture and underlying technologies MSS, HPSS: Tertiarydata storage systems

  6. Evolving ESG to petascale ESG Data System Evolution 2006 Early 2009 2011 • Central database • Centralized curated data archive • Time aggregation • Distribution by file transport • No ESG responsibility for analysis • Shopping-cart-oriented web portal • Testbed data sharing • Federated metadata • Federated portals • Unified user interface • Selected server-side analysis • Location independence • Distributed aggregation • Manual data sharing • Manual publishing • Full data sharing (add to testbed…) • Synchronized federation • metadata, data • Full suite of server-sideanalysis • Model/observation integration • ESG embedded into desktop productivity tools • GIS integration • Model intercomparison metrics • User support, life cycle maintenance CSSM, IPCC,satellite, In situ biogeochemistry,ecosystems CCSMIPCC ESG Data Archive Terabytes Petabytes

  7. data provider data user browser rich client browser rich client ESG Gateway (IPCC) centralizedsecurity services centralizedmetrics services userregistration securityservices monitoringservices metadata services notificationservices services startup/shutdown ESG Gateway (NCAR) ESG Gateway (CCES) OPeNDAP/OLFS(aggregation) product server publishing(harvester) storagemanagement backend analysisand vis engine workflow ESGnode ESGnode ESGnode ESGnode ESGnode ESGnode ESG-CETarchitecture metricsservices replica location services replicamanagement client service API serviceimplementation ESG Node (GFDL) accesscontrol HTTP/FTP/GFTPservers metricsservices backend analysisand vis engine ? publishing(extraction) OPeNDAP/OLFS OPeNDAP/BS monitoringinfo provider ARS mandatorycomponent storagemanagement deeparchivedata JOIN diskcache onlinedata FEDERATION Architecture of thenext-generation ESG • Petascale data archives • Broader geographical distribution of archives • across the United States • around the world • Easy federation of sites • Increased flexibility and robustness

  8. Climate Data Repository and ESG participant ESG participant The team & sponsors National Oceanicand AtmosphericAdministration/PMEL National Center forAtmospheric Research Argonne National Laboratory Lawrence BerkeleyNational Laboratory Lawrence LivermoreNational Laboratory/PCMDI Oak RidgeNational Laboratory USC InformationScience Institute Los Alamos National Laboratory

  9. For more information… ORNL booth at SC2006 • David Bernholdt Other booths at SC2006 • ANL Ian Foster, Veronika Nefedova • Global Grid Forum Ann Chervenak, Carl Kesselman • LBNL Arie Shoshani, Alex Sim • NCAR Luca Cinquini, Jose Garcia, Don Middleton Internet • http://www.earthsystemgrid.org • Esg-manage@earthsystemgrid.org 9 Bernholdt_ESG_0611

More Related