the earth system grid turning climate datasets into community resources n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
The Earth System Grid: Turning Climate Datasets into Community Resources PowerPoint Presentation
Download Presentation
The Earth System Grid: Turning Climate Datasets into Community Resources

Loading in 2 Seconds...

play fullscreen
1 / 9

The Earth System Grid: Turning Climate Datasets into Community Resources - PowerPoint PPT Presentation


  • 92 Views
  • Uploaded on

The Earth System Grid: Turning Climate Datasets into Community Resources. David E. Bernholdt, ORNL on behalf of the Earth System Grid team at Argonne National Laboratory Lawrence Berkeley National Laboratory Lawrence Livermore National Laboratory Los Alamos National Laboratory

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'The Earth System Grid: Turning Climate Datasets into Community Resources' - jerzy


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
the earth system grid turning climate datasets into community resources

The Earth System Grid: Turning Climate Datasets into Community Resources

David E. Bernholdt, ORNL

on behalf of the Earth System Grid team at

Argonne National Laboratory Lawrence

Berkeley National Laboratory

Lawrence Livermore National Laboratory

Los Alamos National Laboratory

National Center for Atmospheric Research

National Oceanic and Atmospheric Administration

Oak Ridge National Laboratory

University of Southern California

www.earthsystemgrid.org

the growing importance of climate simulation data
The growing importance of climate simulation data

Results from the Parallel Climate Model (PCM) depicting wind vectors, surface pressure, sea surface temperature, and sea ice concentration. Prepared from data published in the ESG using the FERRET analysis tool by Gary Strand, NCAR.

  • DOE invests broadly in climate change research:
    • Development of climate models
    • Climate change simulation
    • Model intercomparisons
    • Observational programs
  • Climate change research is increasingly data-intensive:
    • Analysis and intercomparison of simulation and observations from many sources
    • Data used by model developers, impacts analysts, policymakers

2 Bernholdt_ESG_SC07

earth system grid objectives
Earth System Grid objectives

To support the infrastructural needs of the national and international climate community, ESG is providing crucial technology to securely access, monitor, catalog, transport, and distribute data in today’s grid computing environment.

HPChardware running climate models

ESG Portal

ESGSites

3 Bernholdt_ESG_SC07

esg facts and figures
ESG facts and figures

CMIP3 (IPCC AR4) Daily Downloads (through 7/2/07)

Worldwide ESG user base

esg architecture and underlying technologies
Climate data tools

Metadata catalog

NcML (metadata schema)

OPenDAP-G (aggregation and subsetting)

Data management

Data Mover Lite

Storage Resource Manager

Globus toolkit

Globus Security Infrastructure

GridFTP

Monitoring and Discovery Services

Replica Location Service

Security

Access control

MyProxy

User registration

NCAR

Cache

ORNLHPSS

NERSC

NCAR

MSS

RLS

SRM

RLS

SRM

SRM

LANL

Cache

RLS

Data

Search

User

Registration

Catalogs

Browsing

Access

Control

Climate

Metadata

Data

Download

Data

Subsetting

Data

Publishing

Usage

Metrics

SRM

LLNL

Cache

Monitoring

Services

RLS

Web

Browser

Web

Browser

Data

User

Data

Provider

DML

ESG architecture and underlying technologies

First Generation ESG Architecture

search

browse

download

publish

DISK

Cache

MyProxy

OPeNDAP-G

SRM

SRM

ESG Web Portal

RLS

MSS, HPSS: Tertiarydata storage systems

evolving esg to petascale
Evolving ESG to petascale

ESG Data System Evolution

2006

Early 2009

2011

  • Central database
  • Centralized curated data archive
  • Time aggregation
  • Distribution by file transport
  • No ESG responsibility for analysis
  • Shopping-cart-oriented web portal
  • Testbed data sharing
  • Federated metadata
  • Federated portals
  • Unified user interface
  • Selected server-side analysis
  • Location independence
  • Distributed aggregation
  • Manual data sharing
  • Manual publishing
  • Full data sharing (add to testbed…)
  • Synchronized federation
    • metadata, data
  • Full suite of server-sideanalysis
  • Model/observation integration
  • ESG embedded into desktop productivity tools
  • GIS integration
  • Model intercomparison metrics
  • User support, life cycle maintenance

CSSM, IPCC,satellite, In situ biogeochemistry,ecosystems

CCSMIPCC

ESG Data Archive

Terabytes

Petabytes

architecture of the next generation esg

Distribution

Online

Data

Deep

Archives

Distribution

Online

Data

CPU

ESG Node

ESG Node

ESG Gateway (CCES)

ESG Node

ESG Node

Web Portal

Interfaces

Applications

Data &

Metadata

Holdings

ESG Node

ESG Node

ESG Node

Architecture of thenext-generation ESG

Second Generation ESG Architecture

Remote Application Clients

(CDAT, NCL, Ferret, GIS, Publishing, OPeNDAP, DML, Modeling, etc.)

Browser Clients

  • Petascale data archives
  • Broader geographical distribution of archives
    • across the United States
    • around the world
  • Easy federation of sites
  • Increased flexibility and robustness

Web Portals

Local, Remote, and Web Services Interfaces

Cross-Cutting Concerns

(security, logging, monitoring)

Workflow & Orchestration

Applications Components

(data transfer, data publishing, search, analysis, visualization, post-processing, computation)

ESG Gateway (CCSM)

Web Portal

Interfaces

Applications

Data &

Metadata

Holdings

ESG Gateway (IPCC)

Web Portal

Interfaces

Applications

Data &

Metadata

Holdings

Federated ESG Deployment

the team and sponsors

Climate Data Repository and ESG participant

ESG participant

The team and sponsors

National Oceanic& AtmosphericAdministration/PMEL

National Center forAtmospheric Research

Argonne National Laboratory

Lawrence BerkeleyNational Laboratory

Lawrence LivermoreNational Laboratory/PCMDI

Oak RidgeNational Laboratory

USC InformationScience Institute

Los Alamos National Laboratory

for more information
For more information…

ORNL booth at SC2007

  • David Bernholdt

Other booths at SC2007

  • ANL/Global Grid Forum (Booth 551) Ann Chervenak
  • LBNL (351) Arie Shoshani, Alex Sim
  • NCAR (361) Don Middleton

Internet

  • http://www.earthsystemgrid.org
  • Esg-manage@earthsystemgrid.org

9 Bernholdt_ESG_SC07