A services oriented architecture for water resources data
Download
1 / 55

A Services Oriented Architecture for Water Resources Data - PowerPoint PPT Presentation


  • 136 Views
  • Uploaded on

A Services Oriented Architecture for Water Resources Data. David R. Maidment Center for Research in Water Resources University of Texas at Austin. A Services Oriented Architecture for Water Resources Data. CUAHSI and WATERS WaterML and WaterOneFlow Hydrologic Information Server

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' A Services Oriented Architecture for Water Resources Data' - ashton


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
A services oriented architecture for water resources data

A Services Oriented Architecture for Water Resources Data

David R. Maidment

Center for Research in Water Resources

University of Texas at Austin


A services oriented architecture for water resources data1
A Services Oriented Architecture for Water Resources Data

  • CUAHSI and WATERS

  • WaterML and WaterOneFlow

  • Hydrologic Information Server

  • Modeling services


A services oriented architecture for water resources data2
A Services Oriented Architecture for Water Resources Data

  • CUAHSI and WATERS

  • WaterML and WaterOneFlow

  • Hydrologic Information Server

  • Modeling services


What is cuahsi
What is CUAHSI?

UCAR

  • CUAHSI –Consortium of Universities for the Advancement of Hydrologic Science, Inc

  • Formed in 2001 as a legal entity

  • Program office in Washington (5 staff)

  • NSF supports CUAHSI to develop infrastructure and services to advance hydrologic science in US universities

Unidata

Atmospheric

Sciences

Earth

Sciences

Ocean

Sciences

CUAHSI

National Science Foundation

Geosciences Directorate

HIS


Cuahsi member institutions
CUAHSI Member Institutions

112 US Universities as of September 2007


Waters network information system
WATERS Network Information System

Minnehaha Creek

HIS

Team

WATERS

Testbed

NSF has funded work at 11 testbed sites,

each with its own science agenda. HIS supplies the

common information system


His team and its cyberinfrastructure partners
HIS Team and its Cyberinfrastructure Partners

Government:

USGS, EPA,

NCDC

Industry:

ESRI, Kisters,

Microsoft

CUAHSI HIS

HIS Team:

Texas, SDSC,

Utah, Drexel,

Duke

Domain

Sciences:

Unidata, NCAR

LTER, CZEN

GEON

Super

Computer

Centers:

NCSA,

TACC


His waters and the cuahsi community
HIS, WATERS and the CUAHSI Community

Government:

USGS, EPA,

NCDC

Industry:

ESRI, Kisters,

Microsoft

CUAHSI HIS

WATERS Network Information System

HIS

Team

WATERS

Testbed

Domain

Sciences:

Unidata, NCAR

LTER,

GEON

Super

computer

Centers:

NCSA,

TACC


International partners

Government:

USGS, EPA,

NCDC,

USDA

Industry:

ESRI, Kisters,

OpenMI

CUAHSI HIS

WATERS Network

Information System

HIS

Team

WATERS Testbed

Domain

Sciences:

Unidata, NCAR

LTER,

GEON

Super

computer

Centers:

NCSA,

TACC

International Partners

European Commission

Water database design and

model integration (OpenMI)

CSIRO and Bureau of Meteorology

Australian Water Resources Information System


His goals
HIS Goals

  • Hydrologic Data Access – providing better access to a large volume of high quality hydrologic data across the nation

  • Support for Observatories – integrating local observations by academic investigators with hydrologic data for a region

  • Advancement of Hydrologic Science – modeling and analysis of “hydrology in a dynamic earth”

  • Hydrologic Education – bringing more data into the classroom


A services oriented architecture for water resources data3
A Services Oriented Architecture for Water Resources Data

  • CUAHSI and WATERS

  • WaterML and WaterOneFlow

  • Hydrologic Information Server

  • Modeling services


Definition

The CUAHSI Hydrologic Information System (HIS) is a geographically distributed network of data sources and functions that are integrated using web services so that they operate as a connected whole.


Services oriented architecture
Services Oriented Architecture

(from Wikipedia)

  • Service-oriented Architecture (SOA) is an architectural design pattern that concerns itself with defining loosely-coupled relationships between producers and consumers.

  • A major focus of Web services is to make functional building blocks accessible over standard Internet protocols that are independent from platforms and programming languages.

  • The Web Services Description Language (WSDL, pronounced 'wiz-dəl' or spelled out, 'W-S-D-L') is an XML-based language that provides a model for describing Web services.

Defined by the World Wide Web Consortium (W3C)


Web pages and web services
Web Pages and Web Services

http://www.safl.umn.edu/

Uses Hypertext Markup Language (HTML)

Uses WaterML

(an eXtended Markup Language for water data)


Water Data

Water quantity and quality

Soil water

Rainfall & Snow

Modeling

Meteorology

Remote sensing



Nwisweb site output
NWISWeb site output

# agency_cd Agency Code

# site_no USGS station number

# dv_dt date of daily mean streamflow

# dv_va daily mean streamflow value, in cubic-feet per-second

# dv_cd daily mean streamflow value qualification code

#

# Sites in this file include:

# USGS 02087500 NEUSE RIVER NEAR CLAYTON, NC

#

agency_cd site_no dv_dt dv_va dv_cd

USGS 02087500 2003-09-01 1190

USGS 02087500 2003-09-02 649

USGS 02087500 2003-09-03 525

USGS 02087500 2003-09-04 486

USGS 02087500 2003-09-05 733

USGS 02087500 2003-09-06 585

USGS 02087500 2003-09-07 485

USGS 02087500 2003-09-08 463

USGS 02087500 2003-09-09 673

USGS 02087500 2003-09-10 517

USGS 02087500 2003-09-11 454

Time series of

streamflow at a

gaging station

USGS has committed

to supporting CUAHSI’s

GetValues function


Observation stations
Observation Stations

Map for the US

Ameriflux Towers (NASA & DOE)

NOAA Automated Surface Observing System

USGS National Water Information System

NOAA Climate Reference Network


Water Quality Measurement Sites in EPA Storet

Substantial variation in data availability from states

Data from Bora Beran, Drexel University


Water Quality Measurement Sites from

Texas Commission for Environmental Quality (TCEQ)



Observations catalog
Observations Catalog

Specifies what variables are measured at each site, over what time interval,

and how many observations of each variable are available


Point observations information model
Point Observations Information Model

http://www.cuahsi.org/his/webservices.html

USGS

Data Source

GetSites

Streamflow gages

Network

GetSiteInfo

Neuse River near Clayton, NC

Sites

GetVariables

GetVariableInfo

Discharge, stage

(Daily or instantaneous)

Variables

GetValues

Values

206 cfs, 13 August 2006

{Value, Time, Qualifier, Offset}

  • A data source operates an observation network

  • A network is a set of observation sites

  • A site is a point location where one or more variables are measured

  • A variable is a property describing the flow or quality of water

  • A value is an observation of a variable at a particular time

  • A qualifier is a symbol that provides additional information about the value

  • An offset allows specification of measurements at various depths in water


Waterml and wateroneflow

Locations

Variable Codes

Date Ranges

WaterML and WaterOneFlow

STORET

Data

GetSiteInfo

GetVariableInfo

GetValues

Data

NAM

NWIS

WaterML

Data

WaterOneFlow

Web Service

Data

Repositories

Client

EXTRACT

TRANSFORM

LOAD

WaterML is an XML language for communicating water data

WaterOneFlow is a set of web services based on WaterML


Wateroneflow

Set of query functions

Returns data in WaterML

WaterOneFlow


Data Sources

NASA

Storet

Ameriflux

Unidata

NCDC

Extract

NCAR

NWIS

Transform

CUAHSI Web Services

Excel

Visual Basic

ArcGIS

C/C++

Load

Matlab

Fortran

Access

Java

Applications

http://www.cuahsi.org/his/

Some operational services


Getvalues for dissolved oxygen at minnehaha creek
GetValues for Dissolved Oxygenat Minnehaha Creek

http://his.safl.umn.edu/SAFLMC/cuahsi_1_0.asmx?WSDL


A services oriented architecture for water resources data4
A Services Oriented Architecture for Water Resources Data

  • CUAHSI and WATERS

  • WaterML and WaterOneFlow

  • Hydrologic Information Server

  • Modeling services


Hydrologic information server

Supports data discovery, delivery and publication

Data discovery – how do I find the data I want?

Map interface and observations catalogs

Metadata based Search

Data delivery – how do I acquire the data I want?

Use web services or retrieve from local database

Data Publication – how do I publish my observation data?

Use Observations Data Model

Hydrologic Information Server





Nhdplus reach catchments 3km 2
NHDPlus Reach Catchments ~ 3km2

Average reach length = 2km

2.3 million reaches for continental US

About 1000 reach catchments in each 8-digit HUC


Reach attributes

Slope

Elevation

Mean annual flow

Corresponding velocity

Drainage area

% of upstream drainage area in different land uses

Stream order

Reach Attributes


Observations catalog1
Observations Catalog

Specifies what variables are measured at each site, over what time interval,

and how many observations of each variable are available


Hydrologic information server1

GetSites

GetSiteInfo

GetVariables

GetVariableInfo

GetValues

Hydrologic Information Server

WaterOneFlow services

DASH – data access system for hydrology

ArcGISServer

Geospatial Data

Observations Data

Microsoft SQLServer Relational Database


Minnehaha creek experimental site
Minnehaha Creek Experimental Site

http://his.safl.umn.edu/DASH/


Dissolved Oxygen (mg/L)

Depth (m)

Water Temperature (°C)

Dissolved Oxygen (% saturation)


Data heterogeneity

Syntactic mediation

Heterogeneity of format

Use WaterML to get data into the same format

Semantic mediation

Heterogeneity of meaning

Each water data source uses its own vocabulary

Match these up with a common controlled vocabulary

Make standard scientific data queries and have these automatically translated into specific queries on each data source

Data Heterogeneity


request

return

return

request

NAWQA

request

return

return

request

NAM-12

request

return

NWIS

request

return

request

return

return

request

NARR

Objective

  • Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them

What we are doing now …..

Michael Piasecki

Drexel University


NAWQA

NWIS

NARR

HODM

Semantic Mediator

What we would like to do …..

GetValues

GetValues

GetValues

GetValues

generic

request

GetValues

GetValues

Michael Piasecki

Drexel University

GetValues

GetValues


Hydroseek http www hydroseek org
Hydroseekhttp://www.hydroseek.org

Supports search by location and type of data across multiple observation networks including NWIS and Storet


A services oriented architecture for water resources data5
A Services Oriented Architecture for Water Resources Data

  • CUAHSI and WATERS

  • WaterML and WaterOneFlow

  • Hydrologic Information Server

  • Modeling services


  • Project sponsored by the European Commission to promote integration of water models within the Water Framework Directive

  • Software standards for model linking

  • Uses model core as an “engine”

  • http://www.openMI.org


Openmi conceptual framework

Time, T

D

Space, L

Variables, V

OpenMI Conceptual Framework

All values are referenced in a what-where-when framework, allowing different data resources or models to communicate data

VALUES

An application of the data cube to integrate simulation models

Jon Goodall, Duke University


Typical model architecture

User interface

Typical model architecture

Model application

Application

User interface + engine

Engine

Simulates a process – flow in a channel

Accepts input

Provides output

Model

An engine set up to represent a particular location e.g. a reach of the Thames

Write

Input data

Run

Read

Engine

Write

Output data


Linking modelled quantities

Rainfall Runoff Model

River Model

Linking modelled quantities


Data transfer at run time

User interface

User interface

Input data

Input data

Rainfall runoff

River

GetValues(..)

Output data

Output data

Data transfer at run time


Models for the processes
Models for the processes

Rainfall(database)

RR

(Sobek-Rainfall

-Runoff)

River

(InfoWorks RS)

Sewer

(Mouse)


Data exchange

3 Rainfall.GetValues

7 RR.GetValues

6 Sewer.GetValues

Data exchange

Rainfall(database)

4

RR

(Sobek-Rainfall

-Runoff)

2RR.GetValues

1 Trigger.GetValues

5

8

River

(InfoWorks-RS)

call

Sewer

(Mouse)

9

data


Coupling the his with models using openmi
Coupling the HIS with Models using OpenMI

Observations

Data Model

WaterOneFlow Web Services

Water Markup Language

ODM

WOF

WaterML

OpenMI

HEC-RAS

MODFLOW

Others

SWAT

HSFP

“academic models”


Linking the his with hec ras via openmi
Linking the HIS with HEC-RAS via OpenMI

Using real-time streamflow from the HIS as a boundary condition for a HEC-RAS simulation.

A HEC-RAS Model

HIS WaterOneFlow

Web Service for

NWIS Realtime

Streamflow

Workflow “trigger”


Soap based web service for models
SOAP-based Web Service for Models

Database

WSDL

Simulation Model

USGS

NWIS

WSDL

HEC-RAS

OpenMI Workflow

In an effort to build cyberinfrastructure for the hydrologic sciences, we are extending OpenMI to utilize models as web services.


Extending openmi for distributed computing
Extending OpenMI for Distributed Computing

Goal: To allow a modeler to create a workflow from OpenMI components that wrap web services.

Connects to remote model via web services

Connects to remote database via web services

Model linkage designed on client machine


Conclusion

The CUAHSI Hydrologic Information System (HIS) is a geographically distributed network of hydrologic data sources and functions that are integrated using web services so that they function as a connected whole.

For more information: http://www.cuahsi.org/his.html


ad