Waters Observatory Network

1. Waters Observatory Network • Observe water systems • Observe in many different contexts and for different purposes • Have a Waters NetworkInformation System for sharing data • Common language for data • Geographic federation of dispersed data sources through web services

2. CUAHSI HIS Partner Institutions Project co-PI in Phase 2 Collaborator in Phase I

3. 10 WATERS Testbed Sites

4. WATERS Network Information System HIS Team WATERS Testbed

5. HIS Team and its Cyberinfrastructure Partners Government: USGS, EPA, NCDC, USDA Industry: ESRI, Kisters, Microsoft CUAHSI HIS HIS Team: Texas, SDSC, Utah, Drexel, Duke Domain Sciences: Unidata, NCAR LTER, GEON Super Computer Centers: NCSA, TACC

6. HIS and WATERS Government: USGS, EPA, NCDC, USDA Industry: ESRI, Kisters, Microsoft CUAHSI HIS WATERS Network Information System HIS Team WATERS Testbed Domain Sciences: Unidata, NCAR LTER, GEON Super Computer Centers: NCSA TACC

7. CUAHSI Member Institutions 116 Universities as of November 2006

8. HIS, WATERS and the CUAHSI Community Government: USGS, EPA, NCDC, USDA Industry: ESRI, Kisters, OpenMI CUAHSI HIS WATERS Network Information System HIS Team WATERS Testbed Domain Sciences: Unidata, NCAR LTER, GEON Super computer Centers: NCSA, TACC

9. KISTERS Water Resources Management Data System Stefan Fuest KISTERS is building CUAHSI web services onto their database

10. Government: USGS, EPA, NCDC, USDA Industry: ESRI, Kisters, OpenMI CUAHSI HIS WATERS Network Information System HIS Team WATERS Testbed Domain Sciences: Unidata, NCAR LTER, GEON Super computer Centers: NCSA, TACC International Partners European Commission Water database design and model integration (HarmonIT and OpenMI) CSIRO Land and Water Resources Water Resources Observations Network (WRON)

13. HIS Goals • Hydrologic Data Access System – better access to a large volume of high quality hydrologic data • Support for Observatories – synthesizing hydrologic data for a region • Advancement of Hydrologic Science – data modeling and advanced analysis • Hydrologic Education – better data in the classroom, basin-focused teaching

14. Water Data Water quantity and quality Soil water Rainfall & Snow Modeling Meteorology Remote sensing

15. Water Data Web Sites

16. NWISWeb site output # agency_cd Agency Code # site_no USGS station number # dv_dt date of daily mean streamflow # dv_va daily mean streamflow value, in cubic-feet per-second # dv_cd daily mean streamflow value qualification code # # Sites in this file include: # USGS 02087500 NEUSE RIVER NEAR CLAYTON, NC # agency_cd site_no dv_dt dv_va dv_cd USGS 02087500 2003-09-01 1190 USGS 02087500 2003-09-02 649 USGS 02087500 2003-09-03 525 USGS 02087500 2003-09-04 486 USGS 02087500 2003-09-05 733 USGS 02087500 2003-09-06 585 USGS 02087500 2003-09-07 485 USGS 02087500 2003-09-08 463 USGS 02087500 2003-09-09 673 USGS 02087500 2003-09-10 517 USGS 02087500 2003-09-11 454 Time series of streamflow at a gaging station USGS has committed to supporting CUAHSI’s GetValues function

17. Observation Stations Map for the US Ameriflux Towers (NASA & DOE) NOAA Automated Surface Observing System USGS National Water Information System NOAA Climate Reference Network

18. Water Quality Measurement Sites in EPA Storet Substantial variation in data availability from states Data from Bora Beran, Drexel University

19. Water Quality Measurement Sites from Texas Commission for Environmental Quality (TCEQ)

20. Geographic Integration of Storet and TCEQ Data in HIS

21. Observations Catalog Specifies what variables are measured at each site, over what time interval, and how many observations of each variable are available

22. Point Observations Information Model USGS Data Source Streamflow gages Network Neuse River near Clayton, NC Sites Discharge, stage (Daily or instantaneous) Variables Values 206 cfs, 13 August 2006 {Value, Time, Qualifier} • A data source operates an observation network • A network is a set of observation sites • A site is a point location where one or more variables are measured • A variable is a property describing the flow or quality of water • A value is an observation of a variable at a particular time • A qualifier is a symbol that provides additional information about the value

23. Data Source Network Sites Variables Values Data Discovery and Delivery HIS Server Observations Catalog Observations metadata Data Discovery Observations data Web services Data Delivery • HIS facilitates data discovery by building and maintaining observations catalogs • Data delivery occurs through web services from remote data archives or local observations databases. Water resource agencies support data delivery services.

24. Data Sources NASA Storet Ameriflux Unidata NCDC Extract NCAR NWIS Transform CUAHSI Web Services Excel Visual Basic ArcGIS C/C++ Load Matlab Fortran Access Java Applications http://www.cuahsi.org/his/ Some operational services

25. Hydrologic Information System Service Oriented Architecture NASA Web portal Interface (HDAS) Information input, display, query and output services Preliminary data exploration and discovery. See what is available and perform exploratory analyses Storet Ameriflux Unidata NCDC NCAR NWIS 3rd party servers Web services interface CUAHSI Web Services Excel Visual Basic e.g. USGS, NCDC GIS ArcGIS C/C++ Matlab Observatory servers Workgroup HIS IDL SDSC HIS servers Matlab Fortran Splus, R Access SAS D2K, I2K Programming (Fortran, C, VB) Downloads Uploads Ilya Zaslavsky HTML -XML Data access through web services WaterOneFlow Web Services WSDL - SOAP Data storage through web services

26. SOAP Messages Reference a web service, and call a WS method Web service description language WSDL Your Web application Reference a web service, and call a WS method My server Your desktop application SOAP: Simple Object Access Protocol Web services: key terms Web ServiceCode

27. - From different database structures, data collection procedures, quality control, access mechanisms  to uniform signatures … Water Markup Language - Tested in different environments - Standards-based - Can support advanced interfaces via harvested catalogs - Accessible to community - Templates for development of new services - Optimized, error handling, memory management, versioning, run from fast servers And: working with agencies on setting up services!

28. Web service testing pages http://water.sdsc.edu/webservicetestpage/

29. For Developers and Information Managers • Code developers • HIS Team System: code repository, web services • Design Documents • Workbook for using web services in different environments • Web service testing and versioning guidelines and modules • Support • Site managers • Step-by-step instructions for setting up the server (eventually, will be an “HIS appliance”) • Configuration and data management tools What it takes to setup and maintain HIS server, and how to get there from your current setup – THIS AFTERNOON

30. CUAHSI Hydrologic Information System Levels National HIS – San Diego Supercomputer Center Map interface, observations catalogs and web services for national data sources HIS Server Workgroup HIS – research center or academic department Map interface, observations catalogs and web services for regional data sources; observations databases and web services for individual investigator data Personal HIS – an individual hydrologic scientist HIS Analyst Application templates and HydroObjects for direct ingestion of data into analysis environments: Excel, ArcGIS, Matlab, programming languages; MyDB for storage of analysis data

31. Data Ingestion Tim Whiteaker HIS Server Observations Data WaterOneFlow Services HydroObjects ArcGIS Excel Matlab Java .Net HIS Analyst

32. Tutorials and Templates that show how to use Web services in various applications Includes HydroObjects Library ftp://ftp.crwr.utexas.edu/pub/outgoing/CUAHSI/HIS_workbook/20061115/

33. HIS Analyst Demo (Tim Whiteaker) • Ingestion of data using web services into Excel and VB.Net

34. CUAHSI Hydrologic Data Access System http://river.sdsc.edu/HDAS NCDC NASA EPA NWS Observatory Data USGS A common data window for accessing, viewing and downloading hydrologic information

35. Supports data discovery, delivery and publication Data discovery – how do I find the data I want? Map interface and observations catalogs Data delivery – how do I acquire the data I want? Use web services or retrieve from local database Data Publication – how do I publish my observation data? Use Observations Data Model HIS Server

36. Map front end – ArcGIS Server 9.2 (being programmed by ESRI Water Resources for CUAHSI – led by Dean Djokic, an IHE graduate) Relational database – SQL/Server 2005 or Express Web services library – VB.Net programs accessed as a Web Service Description Language (WSDL) HIS Server Architecture

37. HIS Server Demo (Dean Djokic) ESRI Water Resources Applications Group, Redlands CA (a voluntary contribution to CUAHSI)

38. Digital Watershed How can hydrologists integrate observed and modeled data from various sources into a single description of the environment?

39. Digital Watershed Hydrologic Observation Data Geospatial Data (GIS) (Relational database) Digital Watershed Weather and Climate Data Remote Sensing Data (NetCDF) (EOS-HDF) A digital watershed is a synthesis of hydrologic observation data, geospatial data, remote sensing data and weather and climate data into a connected database for a hydrologic region

40. HIS Servers Hydrologic Observations Server GIS Data Server Digital Watershed Weather and Climate Server Remote Sensing Server HIS Servers provide hydrologic observations, weather and climate, GIS and remote sensing data. For HIS version 1.0, the focus isa hydrologic observations server for data from gages and monitoring sites at point locations.

41. Water Resource Regions and HUC’s

42. NHDPlus for Region 17E

43. NHDPlus Reach Catchments ~ 3km2 Average reach length = 2km 2.3 million reaches for continental US About 1000 reach catchments in each 8-digit HUC

44. National Land Cover Dataset http://landcover.usgs.gov/nationallandcover.html Get the data: http://seamless.usgs.gov/

45. Slope Elevation Mean annual flow Corresponding velocity Drainage area % of upstream drainage area in different land uses Stream order Reach Attributes

46. 1:250,000 Scale Soil Information http://www.ncgc.nrcs.usda.gov/products/datasets/statsgo/

47. Hydrologic Landscape Regions USGS Characterization of nature of soil and groundwater systems http://water.usgs.gov/lookup/getspatial?hlrus

48. Arc Hydro An ArcGIS data model for water resources Arc Hydro toolset for implementation Framework for linking hydrologic simulation models Arc Hydro: GIS for Water Resources The Arc Hydro data model and application tools are in the public domain Published in 2002

49. Arc Hydro Links Space and Time Integrate geospatial data for land and water systems All Hydrofeatures of a testbed area will have a unique HydroID Link space and time with HydroID You store this information in a relational database or get it from web services Land base from NHDPlus + added data layers

50. Dan Ames, Idaho State University Open Hydro adopts the Arc Hydro data model and exposes it using open source GIS and database tools. Open Hydro also includes a suite of tools for working with hydrologic data and managing the associated hydro geodatabase.