Ges disc opendap overview
Download
1 / 12

GES DISC OPeNDAP Overview - PowerPoint PPT Presentation


GES DISC OPeNDAP Overview. Aura Data Systems Working Group James Johnson Sept. 27, 2010. OPeNDAP Introduction. OPeNDAP: Open-source Project for a Network Data Access Protocol Evolved from DODS (Distributed Oceanographic Data System)

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha

Download Presentation

GES DISC OPeNDAP Overview

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


GES DISC OPeNDAP Overview

Aura Data Systems Working Group

James Johnson

Sept. 27, 2010


OPeNDAP Introduction

  • OPeNDAP: Open-source Project for a Network Data Access Protocol

  • Evolved from DODS (Distributed Oceanographic Data System)

  • A framework that simplifies all aspects of scientific data networking

  • Protocol based on HTTP

  • Current specification is DAP 2.0; a NASA community standard (see http://www.esdswg.org/spg/rfc/ese-rfc-004)

  • Standards describe

    • encapsulating structured data

    • annotating data with attributes

    • adding semantics that describe the data

  • Formats netCDF(3/4), HDF(4/5 + EOS), binary & user defined

  • Free software (http://opendap.org)

Aura Data Systems Working Group


OPeNDAP Basics

  • OPeNDAP uses the client-server model

  • Server (Hyrax 4.0) consists of a set of CGI scripts specific to the format of the dataset it serves

  • Server fetches a subset of data from the dataset, converts it to a binary format, packages it with some descriptive information, and sends it to the client.

  • Client sends a data request to a server

  • Request is via HTTP using an enhanced URL

  • Client translates the data sent by the server into the specified data format used by the application

Aura Data Systems Working Group


OPeNDAP enhanced URL

Parts of an OPeNDAP URL (without a constraint expression)

>dncview http://dods.gso.uri.edu/cgi-bin/nph-nc/data/fnoc1.nc.das

^ ^ ^ ^ ^ ^ ^

| | | | | | |

Program | | | | | |

Protocol-- | | | | |

Machine Name----- | | | |

Server------------------------------------ | | |

Directory---------------------------------------- | |

Filename---------------------------------------------- |

URL Suffix----------------------------------------------------|

Aura Data Systems Working Group


OPeNDAP URL constraint

  • A constraint expression is appended to the target URL following a question mark, as in the following examples:

    • http://oceans.univ.edu/cgi-bin/nc/expl/buoys.nc?temp

    • http://oceans.univ.edu/cgi-bin/nc/expl/buoys.nc?temp[1,100,5]

    • http://oceans.univ.edu/cgi-bin/nc/expl/buoys.nc?u&lat>15.0

    • http://oceans.univ.edu/cgi-bin/nc/expl/buoys.nc?cast.02<15.0

    • http://oceans.univ.edu/cgi-bin/nc/expl/buoys.nc?station&station.temp<15.0

  • A constraint expression consists of two parts: a projection and a selection, separated by an ampersand (&). Either part may contain several sub-expressions. Either part may be present, or both.

    proj_1,proj_2,...,proj_n&sel_1&sel_2&...&sel_m

Aura Data Systems Working Group


OPeNDAP Example Clients

  • Below is a partial list of OPeNDAP Clients:

    • IDL

    • Matlab

    • GrADS

    • Ferret

    • IDV

    • Panoply

    • CDAT

    • Pydods

    • ncview

    • Generic Web Browser

Aura Data Systems Working Group


OPeNDAP GES DISC Datasets

  • GES DISC serves data following through OPeNDAP:

    • OMI Level 3 (v3 OMAEROe, OMDOAO3e, OMNO2e, OMTO3e, OMAERUVd, OMTO3d, OMUVBd, OMLER)

    • OMI Level 2G (v3 OMAEROG, OMAERUVG, OMCLDO2G, OMCLDRRG, OMDOAO3G, OMHCHOG, OMNO2G, OMSO2G, OMTO3G, OMUVBG)

    • MLS Level 2 (v2 & v1 standard products)

    • HIRDLS Level 2

    • AIRS Level 3 (v5 daily, 8-day, monthly)

    • MODIS Aqua & Terra Level 3 (v5.1 MYD08, MOD08)

    • TOMS Level 3 (v8 Nimbus7, Meteor3 and EP)

Aura Data Systems Working Group


OPeNDAP with Web Browser

List of MLS data files, with response links

Goto http://acdisc.sci.gsfc.nasa.gov/opendap/

Then click on HDF-EOS5, Aura_MLS_Level2, product ( ML2H2O.002) and year (2010)

dds response lists variables with type and sizes

das also includes the attributes

ddx is like das but in XML

info is in a friendly text layout

html is the web interface to subset data

Aura Data Systems Working Group


OPeNDAP with Web Browser (cont.)

Select variables and optionally subset by dimension index. Then get data via ASCII, NetCDF(3) or Binary (DAP) format

URL = http://acdisc.sci.gsfc.nasa.gov/opendap

/HDF-EOS5/Aura_MLS_Level2/ML2H2O.002/2010/

MLS-Aura_L2GP-H2O_v02-23-c01_2010d263.he5.ascii?

L2gpPrecision[1329:1:1329][6:1:39],L2gpValue[1329:1:1329][6:1:39],

lat[1329:1:1329],lon[1329:1:1329],lev[6:1:39]

Aura Data Systems Working Group


OPeNDAP IDL client

IDL> dlm_register, '/usr/local/lib/libidl_opendap.dlm'

IDL> dlm_load, 'opendap‘

IDL> url = 'http://acdisc.sci.gsfc.nasa.gov/opendap/HDF-EOS5/Aura_MLS_Level2/ML2CLO.002/2010/MLS-Aura_L2GP-ClO_v02-23-c01_2010d263.he5’

IDL> status = opendap_get(url, das, mode=‘das’)

IDL> help, /structure, das

** Structure <8f288b4>, 16 tags, length=708, data length=708, refs=1:

NC_GLOBAL STRUCT -> <Anonymous> Array[1]

CONVERGENCE STRUCT -> <Anonymous> Array[1]

L2GPPRECISION STRUCT -> <Anonymous> Array[1]

L2GPVALUE STRUCT -> <Anonymous> Array[1]

QUALITY STRUCT -> <Anonymous> Array[1]

STATUS STRUCT -> <Anonymous> Array[1]

CHUNKNUMBER STRUCT -> <Anonymous> Array[1]

LAT STRUCT -> <Anonymous> Array[1]

LINEOFSIGHTANGLE STRUCT -> <Anonymous> Array[1]

LOCALSOLARTIME STRUCT -> <Anonymous> Array[1]

LON STRUCT -> <Anonymous> Array[1]

ORBITGEODETICANGLE STRUCT -> <Anonymous> Array[1]

LEV STRUCT -> <Anonymous> Array[1]

SOLARZENITHANGLE STRUCT -> <Anonymous> Array[1]

TIME STRUCT -> <Anonymous> Array[1]

DODS__HDFEOS_INFORMATION_COREMETADATA_0

STRUCT -> <Anonymous> Array[1]

IDL> help, /structure, das.l2gpvalue

** Structure <91d8164>, 6 tags, length=56, data length=56, refs=2:

_FILLVALUE FLOAT -999.990

TITLE STRING ‘H2O'

UNITS STRING 'vmr'

MISSING_VALUE FLOAT -999.990

UNIQUEFIELDDEFINITION STRING ‘HIRDLS-MLS-TES-Shared'

COORDINATES STRING 'lat lev‘

Aura Data Systems Working Group


OPeNDAP IDL client (cont.)

IDL> constraint = 'L2gpValue[1329][6:39],L2gpPrecision[1329][6:39],lat[1329],lon[1329],lev[6:39]’

IDL> stat = opendap_get(url, data, ce=constraint)

IDL> help, /structure, data

** Structure <9248a9c>, 5 tags, length=296, data length=296, refs=1:

L2GPPRECISION STRUCT -> <Anonymous> Array[1]

L2GPVALUE STRUCT -> <Anonymous> Array[1]

LAT STRUCT -> <Anonymous> Array[1]

LON STRUCT -> <Anonymous> Array[1]

LEV STRUCT -> <Anonymous> Array[1]

IDL> help, /structure, data.l2gpvalue

** Structure <92156f4>, 2 tags, length=92, data length=92, refs=2:

L2GPVALUE FLOAT Array[35]

DIMENSION_NAMES STRING Array[2]

IDL> print,data.l2gpvalue.dimension_names

NLEVELS NTIMES

IDL> print, data.lat.lat, data.lon.lon, data.lev.lev

40.0019

-103.995

316.228 261.016 215.443 177.828 146.780 121.153

100.000 82.5404 68.1292 56.2341 46.4159 38.3119

31.6228 26.1016 21.5443 14.6780 10.0000 6.81292

4.64159 3.16228 2.15443 1.46780 1.00000 0.681292

0.464159 0.316228 0.215443 0.146780 0.100000 0.0464159

0.0215443 0.0100000 0.00464159 0.00215443

IDL> plot_io, data.l2gpvalue.l2gpvalue*1e6, data.lev.lev, xrange=[1,100], yrange = [1000,0.01], $

title = 'ML2H2O.002 2010 day 263 at lat '+strtrim(string(data.lat.lat),2)+' lon '+strtrim(string(data.lon.lon),2), $

xtitle = ‘H2O [ppmv]', ytitle = 'Pressure [hPa]'

Aura Data Systems Working Group


Panoply Client Example

Aura Data Systems Working Group


ad
  • Login