Tools for Accessing Distributed In-situ Data Collections

1. Tools for Accessing Distributed In-situ Data Collections Donald W. Denbo, NOAA/PMEL-University of Washington/JISAO Jason E. Fabritz, NOAA/PMEL-University of Washington/JISAO Bernard J. Kilonsky, Sea Level Center/ADCP Center, University of Hawaii John R. Osborne, OceanAtlas Software Nancy Soreide, NOAA/PMEL L. Charles Sun, NOAA/National Oceanographic Data Center Willa H. Zhu, NOAA/PMEL-University of Washington/JISAO AGU 2002 Western Pacific Geophysics Meeting, Wellington, New Zealand

2. Challenge • The process of locating and selecting individual observations from many on-line, large collections of in-situ data observations is complex • E.g., millions of individual ocean profiles, time series, buoy tracks, etc. AGU 2002 Western Pacific Geophysics Meeting, Wellington, New Zealand

3. Some Answers • A suite of tools focused on in-situ data and metadata: • Locating • Subsetting • Graphing, viewing and inter-comparing • Downloading • Developed for geographically distributed collections of in-situ data • Freely available AGU 2002 Western Pacific Geophysics Meeting, Wellington, New Zealand

4. Java and Open Source • Developed in Java • Tools operate smoothly across a variety of hardware platforms and operating systems. • Reduced development/maintenance costs • Tools are created from widely-used, standards-based, freely available, off-the-shelf, Open Source components. AGU 2002 Western Pacific Geophysics Meeting, Wellington, New Zealand

5. Modular components • SGT Java graphics • Graphical presentation of oceanographic and meteorological data • NdEdit • Selecting and subsetting from large collections of in-situ data • XML-based MetaArchitect • Automated metadata generation • LDAP • Network data discovery based on geographic region, observed quantities or keyword AGU 2002 Western Pacific Geophysics Meeting, Wellington, New Zealand

6. SGT Java graphics toolkit • SGT • A collection of Java classes and interfaces to create graphical applications • Easy-to-use SGT JavaBeans in next release • Features • Contour, line and vector plots, user settable or automatically scaled axes (including time axes), mouse-movable, customizable objects (labels, line keys, icons), automatic generation of legends • Usage • Wide variety of scientific and commercial applications • 9672 downloads by 3548 unique sites from 64 countries AGU 2002 Western Pacific Geophysics Meeting, Wellington, New Zealand

7. SGT Javagraphics examples Two time axis styles

8. NdEditSubsetting observed data from large data collections • Interactive, linked windows for viewing or selecting individual data observations • View in 2D slices (lat, lon, depth/height, time) • Select in box, polygon, section line, along bathymetry contours • Refine data selection by deleting or retaining files • Applications • Java OceanAtlas (profile data) • Climate Data Portal (distributed ocean data) • EPIC (desktop system for in-situ data management) AGU 2002 Western Pacific Geophysics Meeting, Wellington, New Zealand

9. NdEditExplore data locations in linked windows Ocean profiles viewed in linked windows showing lat-lon and lat-time views Built-in or customized bathymetry contourscan be added to lat-lon view

10. NdEditSelect desired subset from a data collection Time window Filter in time window to view or selecta subset of the data

11. MetaArchitectXML service for automated metadata extraction • Automated scanning of data files to extract FGDC compliant metadata to support data discovery • Flexible and configurable XML-based tool • Applications: • netCDF data • NODC data archives • DODS? AGU 2002 Western Pacific Geophysics Meeting, Wellington, New Zealand

12. Using LDAP for environmental data discovery on the Internet • Find data sets on-line based on geographic region, observed quantities or keyword • Automated population of directory directly from data servers • Metadata is instantly known to distributed LDAP directories • Access controls available for experimental data products • Features • Open Source, standards-based, platform independent, scalable, extensible, works with application or web client • Applications • Climate Data Portal (distributed data access) • OceanShare (collaborative tool) • DODS (possible, interested) AGU 2002 Western Pacific Geophysics Meeting, Wellington, New Zealand

13. Applications using the tools • ncBrowse • interactive browsing of data and metadata from DODS or other netCDF data files • Climate Data Portal • centralized access to distributed observing system data and historical archives • Java OceanAtlas • Intensive browsing of ocean profile data • OceanShare • network collaboration tool AGU 2002 Western Pacific Geophysics Meeting, Wellington, New Zealand

14. ncBrowseNetworking data file browser • Flexible, interactive graphical display of data and attributes for wide range of netCDF data file conventions • Reads remote netCDF files • Reads DODS data files • Axis mapping, parametric plots, flexible mapping of independent variables to axes • Line, vector and contour plots • Animations • Usage • Data file browse tool for wide range of disciplines and programs • E.g., Unidata, DODS, WOCE data • 4749 downloads by 2265 unique sites from 55 countries AGU 2002 Western Pacific Geophysics Meeting, Wellington, New Zealand

15. Climate Data PortalAccess to distributed ocean observations data • Data available: • TAO El Nino buoy data (realtime) – PMEL • Global sea level data (realtime) – University of Hawaii • Global Temperature-Salinity Profile Program (GTSPP) – NODC • PMEL hydrographic data and time series collection – PMEL • WOCE Acoustic Doppler Current Profiler data– University of Hawaii • Asia Pacific Data Research Center in-situ data –Intl. Pacific Research Center • US Coastal Ocean Time Series database – NODC • Future plans: WOCE hydrography, WOCE/CLIVAR datasets, MEDS Canada, ARGO profiling floats • Features: • Data discovery services based on time/space/keyword (LDAP), • Graphical data viewing/subsetting (NdEdit), data graphics (SGT) • On-line help, tutorial • Access from Java application or web client • DODS interoperability (pending) AGU 2002 Western Pacific Geophysics Meeting, Wellington, New Zealand

16. Java OceanAtlasfor browsing Ocean profile data • Sophisticated browsing/visualization of profile data • Property-property plots with up to 7 x axes • Offset property-property profiles (waterfall diagrams) • Contour plots/residual contour plots • Map plots with optional "bubble" isosurface plots • Linked browsing of all plot types • Built-in and custom calculations • Browse data locations in space/time (NdEdit) • Select in box, polygon, section line, along bathymetry contours • Refine data selection by deleting or retaining files AGU 2002 Western Pacific Geophysics Meeting, Wellington, New Zealand

17. OceanShareNetwork collaboration tool • Access to data and graphics in a networked collaborative tool environment • Access to local and remote data files, graphics • Standard collaboration tool features (annotation, whiteboard, etc) • Utilizes: CDP, NdEdit, SGT, LDAP, Slide, JavaSpaces, Jini, SpaceChat • Secure Document Repository • Secure location for sharing: • documents, e.g., proposals, papers in progress, etc. • data, images • prior collaborative session state • Access control list for individual documents and/or folders • Document version control • Documents accessible from web and desktop • Utilizes: WebDAV, Slide, Tomcat servlet container AGU 2002 Western Pacific Geophysics Meeting, Wellington, New Zealand

18. Summary • Tools • SGT - Java graphics toolkit • NdEdit – Subsetting ocean observations from collections • MetaArchitect - Automated metadata generation • LDAP – Environmental data directory services • Applications • ncBrowse – Interactive file browser • Climate Data Portal – Distributed ocean data access • Java OceanAtlas – Ocean profile data browsing • OceanShare – Collaboration tool AGU 2002 Western Pacific Geophysics Meeting, Wellington, New Zealand

19. In-situ data tools and applications are freely available • Web • http://www.epic.noaa.gov/epic/software/ep_java.htm • Email • Donald.W.Denbo@noaa.gov AGU 2002 Western Pacific Geophysics Meeting, Wellington, New Zealand