1 / 19

Dapper and Argo

Dapper and Argo. Joe Sirott PMEL/NOAA. What is Dapper?. Web server that provides distributed access to in-situ data via OPeNDAP protocol Clients include GrADS, Matlab, NcBrowse, Java Ocean Atlas Available data: PMEL EPIC database ARGO profiles NODC World Ocean Database 2001

parkerwilliam
Download Presentation

Dapper and Argo

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Dapper and Argo Joe Sirott PMEL/NOAA

  2. What is Dapper? • Web server that provides distributed access to in-situ data via OPeNDAP protocol • Clients include GrADS, Matlab, NcBrowse, Java Ocean Atlas • Available data: • PMEL EPIC database • ARGO profiles • NODC World Ocean Database 2001 • All components are open-source

  3. What does Dapper do? • Efficiently aggregates thousands of in-situ data files (>2000000 for WODB2001) • Handles several in-situ data formats: • Argo GDAC netCDF • PMEL EPIC netCDF • COARDS compliant netCDF • Supports time series or profiles • Supports space/time queries • Based on HPCC funded Climate Data Portal (Soreide, Zhu, Denbo)

  4. What is OPeNDAP? • OPeNDAP (Open source Project for a Network Data Access Protocol) is a protocol for remotely accessing scientific data • Uses HTTP as transport

  5. Dapper OPeNDAP Example (space/time query) • Client HTTP request:http://foo.org/.../dset.cdp.dods?lat,temp,pres&lat>0&lat<20 • Dapper HTTP response:Dataset { Sequence { Float32 lat; Float64 _id; Sequence { Float32 pres,temp; } profile; } } location = { { -4.01,1, { {4.3,26.808}, {100,24.982}, …}, { -52, 2, {{8.3,3.77},{100,3.743},…}

  6. Dapper OPeNDAP Example (single profile) • Client HTTP request:http://foo.org/.../dset.cdp.dods?lat,lon,time,profile&_id=20 • Dapper HTTP response:Dataset { Sequence { Float64 time; Float32 lat,lon; Sequence { Float32 pres,temp; } profile; } } location = { {1034140577999,-52.9,310.144}, {{8.3,3.77},{9.1,3.77},…}};

  7. Dapper architectural overview • Uses relational database to store metadata and coordinate boundaries for each profile/time series file • Data maintained in original format • All data delivered as OPeNDAP sequences

  8. CDP service netCDF service netCDF profile netCDF profile Database loader Dapper architecture OPeNDAP Protocol (HTTP) OPeNDAP clientDChart Dapper HTTP Server CORBA (IIOP) Climate Data Portal Java netCDF library MySQL

  9. DChart • Browser based in-situ OPeNDAP client • Based on FY2004 HPCC funded project for shipboard display of cruise data • AJAX based user interface

  10. DChart Design Goals • Highly responsive browser based user interface • Low learning curve • Rapid previewing and downloading of Dapper datasets • Highly interactive map for region selection • No Java applets or proprietary plug-ins

  11. CDP service netCDF service netCDF profile netCDF profile Database loader Dapper architecture diagram OPeNDAP Protocol (HTTP) OPeNDAP clientDChart Dapper HTTP Server CORBA (IIOP) Climate Data Portal Java netCDF library MySQL

  12. Web Browser XML/JavaScript (HTTP) OPeNDAP Protocol (HTTP) JavaScript DChart Servlet Dapper JFreeChartSGT OPeNDAPClient XMLEngine DChart architecture

  13. DChart DemoNDBC videoArgo video

  14. CDFsync • Rapid synchronization of netCDF in-situ files over a network • Only compressed file differences transmitted across network • Based on rsync, but optimized for netCDF and file lists consisting of thousands or millions of files • Funded by HPCC 2004 (Charles Sun et al) • Available at http://www.epic.noaa.gov/epic/software/cdfsync/

  15. Future plans • Constraints (QC flags, attributes) • Metadata display • Trajectory and section plots • Multiple plot windows • Save/restore Web app state

  16. Questions?

  17. More info PMEL Dapper Server http://dapper.pmel.noaa.gov/dapper PMEL Dapper UI http://dapper.pmel.noaa.gov/dchart Downloads, propaganda http://www.epic.noaa.gov/epic/software/dapper/ Joe.Sirott@noaa.gov

  18. Back

  19. Back

More Related