APDRC...Quo Facis? - PowerPoint PPT Presentation

Apdrc quo facis l.jpg
Download
1 / 34

APDRC...Quo Facis?. Data serving and archiving at the Asia-Pacific Data-Research Center IPRC. The APDRC was established within the IPRC to be a climate data and web-based product serving facility. Our mission is to increase understanding of climate variability in the Asia-Pacific region by:

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

APDRC...Quo Facis?

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Apdrc quo facis l.jpg

APDRC...Quo Facis?

Data serving and archiving at the Asia-Pacific Data-Research Center

IPRC


Slide2 l.jpg

The APDRC was established within the IPRC to be a climate data and web-based product serving facility. Our mission is to increase understanding of climate variability in the Asia-Pacific region by:

  • developing the computational, data-management, and networking infrastructure necessary to make data resources readily accessible and usable by researchers and other users;

  • undertaking data-intensive research activities that will both advance knowledge and lead to improvements in data collection and preparation.


Challenges for distribution of climate data l.jpg

Challenges for Distribution of Climate Data

  • disparate data types

    • station data, gridded data, ocean data, atmospheric data, levels, time-series, etc.

  • disparate data formats

    • netcdf, grib, flat binary, etc.

  • datasets can be large

    • coupled model out, long integrations, high resolution (time/space), etc.

  • users needs are widely varying


Data management and data serving l.jpg

Data Managementand Data Serving

  • metadata (so-called “readme” files)

  • data discovery --> search for data

    • data mining

  • data transport --> getting data to users

    • OPeNDAP, ftp, http, etc.

  • on-line browse --> view data

    • LAS

  • data archive


Data transport l.jpg

Data Transport

  • ftp (limited)

  • direct binary access NFS (internal)

  • OPeNDAP


Dods opendap l.jpg

DODS/OPeNDAP

  • DODS: Distributed Oceanographic Data System, a.k.a OpenDAP: Open Source Project for a Network Data Access Protocol, is a software framework used for data networking.

  • DODS provides tools for making local data accessible to remote locations regardless of local storage format


On line browse l.jpg

On-line Browse

  • LAS (gridded data)

  • EPIC (station data)


Live access server las l.jpg

Live Access Server (LAS)

  • LAS is a Web server designed to provide flexible access to geo-referenced scientific data. LAS enables the Web user to visualize data with on-the-fly graphics

  • request custom subsets of variables in a choice of file formats

  • access background reference material about the data (metadata)

  • compare (difference) variables from distributed locations


Slide11 l.jpg

Serving gridded products


Slide14 l.jpg

Compare

distributed

GODAE model

outputs

Define region

Select time


Slide15 l.jpg

Serving non-gridded products


Data archive l.jpg

Data Archive

  • parallel systems (Linux and Solaris based)

  • similar raid systems

  • remote aggregation

  • data spread across ocean, atmosphere, air-sea flux

  • model reanalysis, forecast, etc.


Slide17 l.jpg

Old paradigm

User access to data either by copy or ftp, then use local software

Data reformat (netcdf, ieee, grib, etc.)?

Client s/w limitations?

Disk storage/transfer limitations?


Slide18 l.jpg

Non-gridded (station) data

Gridded data


Slide19 l.jpg

“sister server”


Slide20 l.jpg

GrADS

Matlab

ferret


Apdrc project support l.jpg

APDRC Project Support

  • Global Ocean Data Assimilation Experiment (GODAE)

  • argo

  • Pacific Regional Integrated Data Enterprise (PRIDE)

  • Regional modeling

  • Ministry of Education, Culture, Sports, Science and Technology (MEXT)

  • JAMSTEC Earth Simulator Group (ESG)

  • Quality control of historical profiles of temperature and salinity for the global oceans (HydroBase2, WHOI) and the Indian Ocean (CSIRO), GTSPP


Godae l.jpg

GODAE

As the international community participates in the demonstration phase of GODAE,

(2003-2005)

there is an overarching need for rapid delivery of data products from satellites and models to the broad user community including regional operational entities.


Godae23 l.jpg

GODAE

  • Operational Systems

    • Naval Oceanographic Office (NAVOCEAN; NRL, Bay St. Louis, MS)

    • Fleet Numerical Meteorology and Oceanography Center (FNMOC; Monterey, CA)

    • National Centers for Environmental Prediction (NCEP)

  • Research and Development Systems

    • Argo

    • NASA Seasonal to Interannual Prediction Project (NSIPP)

    • Estimating the Circulation and Climate of the Ocean (ECCO; Scripps, MIT and JPL)

    • Hybrid Co-ordinate Ocean Model (HYCOM; Miami, NRL)

    • Naval Research Laboratory (NRL)

  • Data

    • National Virtual Ocean Data System (NVODS)


Slide24 l.jpg

EPIC access to argo data

Select subregion

Sort function


Slide25 l.jpg

Further subset


Slide26 l.jpg

Plot specific station

Plot/save along section


Conceptual framework for pride l.jpg

PRIDE

Conceptual Framework forPRIDE

US/NOAA Context

Bi-Laterals with

Australia

Japan

New Zealand

Global/Regional Observations

GCOS/GOOS/IOOS

IPRC

APDRC

Pacific

Services

Center

New NOAA

Facility in

Hawaii

Pacific Climate Information

System

WMO/NOAA RCCs

RISA

NOAA Data

Centers


Future thoughts what does this mean for clivar l.jpg

Future thoughts;what does this mean for CLIVAR?

  • Users

    • Requirements?

      • Data discovery, data transport?

    • Regional analysis of global models

  • Collaborators

    • production of data products

    • communication of results to wider audience

  • include model output as data products?

    • metadata


Issues cont d l.jpg

Issues, cont’d

  • Who does this?

  • Key is to determine users and uses

  • Model interpretation; data providers=data users?

  • Development of data products

  • Leave details to someone else?


Apdrc customers l.jpg

Non-scientific users:

Plot generation

More co-ordinated approach

Web interface >> data format/server tech

APDRC Customers

Scientific users:

  • direct access to data

  • typically large (long timeseries and/or large spatial extents

    Server technology >> interface/presentaion


Slide33 l.jpg

TBA

  • Development/identification of data products

  • Use of numerical models

    • Future observational programs

    • Model metrics

  • Use of observations


Slide34 l.jpg

The World Climate Research Program (WCRP) was formed in 1980 by the International Council for Science (ICSU) and the World Meteorological Office (WMO) to provide international framework for scientific cooperation in the study of global change.

The Global Climate Observing System (GCOS) was established in 1992 to ensure that the observations and information needed to address climate-related issues are obtained and made available to all potential users. GCOS is sponsored by the WMO, the ICSU, and the Intergovernmental Oceanographic Commission (IOC).

The Global Ocean Observing System (GOOS) is a sustained, co-ordinated international system for gathering data about the oceans and seas of the Earth, a system for processing such data, with other relevant data from other domains, to enable the generation of beneficial analytical and prognostic environmental information services, and the research and development on which such services depend for their improvement.

In 1978 (?), WCRP, IOC and the Scientific Committee on Oceanic Research (SCOR) formed the Climate Change Committee for the Ocean (CCCO). This committee then established the Ocean Observation Panel for Climate (OOPC).

The OOPC oversees the implementation of an ocean observation system for GCOS (or, equivalently, the climate component for GOOS) and, with GCOS the Atmospheric Observation Panel for Climate (AOPC). OOPC has several projects, including CLIVAR, GODAE and ARGO.

GODAE therefore is a project of OOPC and is sponsored by GCOS, GOOS and WCRP.


  • Login