A services oriented architecture for water resources data
This presentation is the property of its rightful owner.
Sponsored Links
1 / 55

A Services Oriented Architecture for Water Resources Data PowerPoint PPT Presentation


  • 99 Views
  • Uploaded on
  • Presentation posted in: General

A Services Oriented Architecture for Water Resources Data. David R. Maidment Center for Research in Water Resources University of Texas at Austin. A Services Oriented Architecture for Water Resources Data. CUAHSI and WATERS WaterML and WaterOneFlow Hydrologic Information Server

Download Presentation

A Services Oriented Architecture for Water Resources Data

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


A services oriented architecture for water resources data

A Services Oriented Architecture for Water Resources Data

David R. Maidment

Center for Research in Water Resources

University of Texas at Austin


A services oriented architecture for water resources data1

A Services Oriented Architecture for Water Resources Data

  • CUAHSI and WATERS

  • WaterML and WaterOneFlow

  • Hydrologic Information Server

  • Modeling services


A services oriented architecture for water resources data2

A Services Oriented Architecture for Water Resources Data

  • CUAHSI and WATERS

  • WaterML and WaterOneFlow

  • Hydrologic Information Server

  • Modeling services


What is cuahsi

What is CUAHSI?

UCAR

  • CUAHSI –Consortium of Universities for the Advancement of Hydrologic Science, Inc

  • Formed in 2001 as a legal entity

  • Program office in Washington (5 staff)

  • NSF supports CUAHSI to develop infrastructure and services to advance hydrologic science in US universities

Unidata

Atmospheric

Sciences

Earth

Sciences

Ocean

Sciences

CUAHSI

National Science Foundation

Geosciences Directorate

HIS


Cuahsi member institutions

CUAHSI Member Institutions

112 US Universities as of September 2007


Waters network information system

WATERS Network Information System

Minnehaha Creek

HIS

Team

WATERS

Testbed

NSF has funded work at 11 testbed sites,

each with its own science agenda. HIS supplies the

common information system


His team and its cyberinfrastructure partners

HIS Team and its Cyberinfrastructure Partners

Government:

USGS, EPA,

NCDC

Industry:

ESRI, Kisters,

Microsoft

CUAHSI HIS

HIS Team:

Texas, SDSC,

Utah, Drexel,

Duke

Domain

Sciences:

Unidata, NCAR

LTER, CZEN

GEON

Super

Computer

Centers:

NCSA,

TACC


His waters and the cuahsi community

HIS, WATERS and the CUAHSI Community

Government:

USGS, EPA,

NCDC

Industry:

ESRI, Kisters,

Microsoft

CUAHSI HIS

WATERS Network Information System

HIS

Team

WATERS

Testbed

Domain

Sciences:

Unidata, NCAR

LTER,

GEON

Super

computer

Centers:

NCSA,

TACC


International partners

Government:

USGS, EPA,

NCDC,

USDA

Industry:

ESRI, Kisters,

OpenMI

CUAHSI HIS

WATERS Network

Information System

HIS

Team

WATERS Testbed

Domain

Sciences:

Unidata, NCAR

LTER,

GEON

Super

computer

Centers:

NCSA,

TACC

International Partners

European Commission

Water database design and

model integration (OpenMI)

CSIRO and Bureau of Meteorology

Australian Water Resources Information System


His goals

HIS Goals

  • Hydrologic Data Access – providing better access to a large volume of high quality hydrologic data across the nation

  • Support for Observatories – integrating local observations by academic investigators with hydrologic data for a region

  • Advancement of Hydrologic Science – modeling and analysis of “hydrology in a dynamic earth”

  • Hydrologic Education – bringing more data into the classroom


A services oriented architecture for water resources data3

A Services Oriented Architecture for Water Resources Data

  • CUAHSI and WATERS

  • WaterML and WaterOneFlow

  • Hydrologic Information Server

  • Modeling services


A services oriented architecture for water resources data

Definition

The CUAHSI Hydrologic Information System (HIS) is a geographically distributed network of data sources and functions that are integrated using web services so that they operate as a connected whole.


Services oriented architecture

Services Oriented Architecture

(from Wikipedia)

  • Service-oriented Architecture (SOA) is an architectural design pattern that concerns itself with defining loosely-coupled relationships between producers and consumers.

  • A major focus of Web services is to make functional building blocks accessible over standard Internet protocols that are independent from platforms and programming languages.

  • The Web Services Description Language (WSDL, pronounced 'wiz-dəl' or spelled out, 'W-S-D-L') is an XML-based language that provides a model for describing Web services.

Defined by the World Wide Web Consortium (W3C)


Web pages and web services

Web Pages and Web Services

http://www.safl.umn.edu/

Uses Hypertext Markup Language (HTML)

Uses WaterML

(an eXtended Markup Language for water data)


A services oriented architecture for water resources data

Water Data

Water quantity and quality

Soil water

Rainfall & Snow

Modeling

Meteorology

Remote sensing


Water data web sites

Water Data Web Sites


Nwisweb site output

NWISWeb site output

# agency_cd Agency Code

# site_no USGS station number

# dv_dt date of daily mean streamflow

# dv_va daily mean streamflow value, in cubic-feet per-second

# dv_cd daily mean streamflow value qualification code

#

# Sites in this file include:

# USGS 02087500 NEUSE RIVER NEAR CLAYTON, NC

#

agency_cdsite_nodv_dtdv_vadv_cd

USGS020875002003-09-011190

USGS020875002003-09-02649

USGS020875002003-09-03525

USGS020875002003-09-04486

USGS020875002003-09-05733

USGS020875002003-09-06585

USGS020875002003-09-07485

USGS020875002003-09-08463

USGS020875002003-09-09673

USGS020875002003-09-10517

USGS020875002003-09-11454

Time series of

streamflow at a

gaging station

USGS has committed

to supporting CUAHSI’s

GetValues function


Observation stations

Observation Stations

Map for the US

Ameriflux Towers (NASA & DOE)

NOAA Automated Surface Observing System

USGS National Water Information System

NOAA Climate Reference Network


A services oriented architecture for water resources data

Water Quality Measurement Sites in EPA Storet

Substantial variation in data availability from states

Data from Bora Beran, Drexel University


A services oriented architecture for water resources data

Water Quality Measurement Sites from

Texas Commission for Environmental Quality (TCEQ)


A services oriented architecture for water resources data

Geographic Integration of Storet and TCEQ Data in HIS


Observations catalog

Observations Catalog

Specifies what variables are measured at each site, over what time interval,

and how many observations of each variable are available


Point observations information model

Point Observations Information Model

http://www.cuahsi.org/his/webservices.html

USGS

Data Source

GetSites

Streamflow gages

Network

GetSiteInfo

Neuse River near Clayton, NC

Sites

GetVariables

GetVariableInfo

Discharge, stage

(Daily or instantaneous)

Variables

GetValues

Values

206 cfs, 13 August 2006

{Value, Time, Qualifier, Offset}

  • A data source operates an observation network

  • A network is a set of observation sites

  • A site is a point location where one or more variables are measured

  • A variable is a property describing the flow or quality of water

  • A value is an observation of a variable at a particular time

  • A qualifier is a symbol that provides additional information about the value

  • An offset allows specification of measurements at various depths in water


Waterml and wateroneflow

Locations

Variable Codes

Date Ranges

WaterML and WaterOneFlow

STORET

Data

GetSiteInfo

GetVariableInfo

GetValues

Data

NAM

NWIS

WaterML

Data

WaterOneFlow

Web Service

Data

Repositories

Client

EXTRACT

TRANSFORM

LOAD

WaterML is an XML language for communicating water data

WaterOneFlow is a set of web services based on WaterML


Wateroneflow

Set of query functions

Returns data in WaterML

WaterOneFlow


A services oriented architecture for water resources data

Data Sources

NASA

Storet

Ameriflux

Unidata

NCDC

Extract

NCAR

NWIS

Transform

CUAHSI Web Services

Excel

Visual Basic

ArcGIS

C/C++

Load

Matlab

Fortran

Access

Java

Applications

http://www.cuahsi.org/his/

Some operational services


Getvalues for dissolved oxygen at minnehaha creek

GetValues for Dissolved Oxygenat Minnehaha Creek

http://his.safl.umn.edu/SAFLMC/cuahsi_1_0.asmx?WSDL


A services oriented architecture for water resources data4

A Services Oriented Architecture for Water Resources Data

  • CUAHSI and WATERS

  • WaterML and WaterOneFlow

  • Hydrologic Information Server

  • Modeling services


Hydrologic information server

Supports data discovery, delivery and publication

Data discovery – how do I find the data I want?

Map interface and observations catalogs

Metadata based Search

Data delivery – how do I acquire the data I want?

Use web services or retrieve from local database

Data Publication – how do I publish my observation data?

Use Observations Data Model

Hydrologic Information Server


A services oriented architecture for water resources data

Water Resource Regions and HUC’s


National hydrography dataset

National Hydrography Dataset


Nhdplus for region 17e

NHDPlus for Region 17E


Nhdplus reach catchments 3km 2

NHDPlus Reach Catchments ~ 3km2

Average reach length = 2km

2.3 million reaches for continental US

About 1000 reach catchments in each 8-digit HUC


Reach attributes

Slope

Elevation

Mean annual flow

Corresponding velocity

Drainage area

% of upstream drainage area in different land uses

Stream order

Reach Attributes


Observations catalog1

Observations Catalog

Specifies what variables are measured at each site, over what time interval,

and how many observations of each variable are available


Hydrologic information server1

GetSites

GetSiteInfo

GetVariables

GetVariableInfo

GetValues

Hydrologic Information Server

WaterOneFlow services

DASH – data access system for hydrology

ArcGISServer

Geospatial Data

Observations Data

Microsoft SQLServer Relational Database


Minnehaha creek experimental site

Minnehaha Creek Experimental Site

http://his.safl.umn.edu/DASH/


A services oriented architecture for water resources data

Dissolved Oxygen (mg/L)

Depth (m)

Water Temperature (°C)

Dissolved Oxygen (% saturation)


Data heterogeneity

Syntactic mediation

Heterogeneity of format

Use WaterML to get data into the same format

Semantic mediation

Heterogeneity of meaning

Each water data source uses its own vocabulary

Match these up with a common controlled vocabulary

Make standard scientific data queries and have these automatically translated into specific queries on each data source

Data Heterogeneity


A services oriented architecture for water resources data

request

return

return

request

NAWQA

request

return

return

request

NAM-12

request

return

NWIS

request

return

request

return

return

request

NARR

Objective

  • Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them

What we are doing now …..

Michael Piasecki

Drexel University


A services oriented architecture for water resources data

NAWQA

NWIS

NARR

HODM

Semantic Mediator

What we would like to do …..

GetValues

GetValues

GetValues

GetValues

generic

request

GetValues

GetValues

Michael Piasecki

Drexel University

GetValues

GetValues


Hydroseek http www hydroseek org

Hydroseekhttp://www.hydroseek.org

Supports search by location and type of data across multiple observation networks including NWIS and Storet


A services oriented architecture for water resources data5

A Services Oriented Architecture for Water Resources Data

  • CUAHSI and WATERS

  • WaterML and WaterOneFlow

  • Hydrologic Information Server

  • Modeling services


A services oriented architecture for water resources data

  • Project sponsored by the European Commission to promote integration of water models within the Water Framework Directive

  • Software standards for model linking

  • Uses model core as an “engine”

  • http://www.openMI.org


Openmi conceptual framework

Time, T

D

Space, L

Variables, V

OpenMI Conceptual Framework

All values are referenced in a what-where-when framework, allowing different data resources or models to communicate data

VALUES

An application of the data cube to integrate simulation models

Jon Goodall, Duke University


Typical model architecture

User interface

Typical model architecture

Model application

Application

User interface + engine

Engine

Simulates a process – flow in a channel

Accepts input

Provides output

Model

An engine set up to represent a particular location e.g. a reach of the Thames

Write

Input data

Run

Read

Engine

Write

Output data


Linking modelled quantities

Rainfall Runoff Model

River Model

Linking modelled quantities


Data transfer at run time

User interface

User interface

Input data

Input data

Rainfall runoff

River

GetValues(..)

Output data

Output data

Data transfer at run time


Models for the processes

Models for the processes

Rainfall(database)

RR

(Sobek-Rainfall

-Runoff)

River

(InfoWorks RS)

Sewer

(Mouse)


Data exchange

3 Rainfall.GetValues

7 RR.GetValues

6 Sewer.GetValues

Data exchange

Rainfall(database)

4

RR

(Sobek-Rainfall

-Runoff)

2RR.GetValues

1 Trigger.GetValues

5

8

River

(InfoWorks-RS)

call

Sewer

(Mouse)

9

data


Coupling the his with models using openmi

Coupling the HIS with Models using OpenMI

Observations

Data Model

WaterOneFlow Web Services

Water Markup Language

ODM

WOF

WaterML

OpenMI

HEC-RAS

MODFLOW

Others

SWAT

HSFP

“academic models”


Linking the his with hec ras via openmi

Linking the HIS with HEC-RAS via OpenMI

Using real-time streamflow from the HIS as a boundary condition for a HEC-RAS simulation.

A HEC-RAS Model

HIS WaterOneFlow

Web Service for

NWIS Realtime

Streamflow

Workflow “trigger”


Soap based web service for models

SOAP-based Web Service for Models

Database

WSDL

Simulation Model

USGS

NWIS

WSDL

HEC-RAS

OpenMI Workflow

In an effort to build cyberinfrastructure for the hydrologic sciences, we are extending OpenMI to utilize models as web services.


Extending openmi for distributed computing

Extending OpenMI for Distributed Computing

Goal: To allow a modeler to create a workflow from OpenMI components that wrap web services.

Connects to remote model via web services

Connects to remote database via web services

Model linkage designed on client machine


A services oriented architecture for water resources data

Conclusion

The CUAHSI Hydrologic Information System (HIS) is a geographically distributed network of hydrologic data sources and functions that are integrated using web services so that they function as a connected whole.

For more information: http://www.cuahsi.org/his.html


  • Login