servogrid basics l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
SERVOGrid Basics PowerPoint Presentation
Download Presentation
SERVOGrid Basics

Loading in 2 Seconds...

play fullscreen
1 / 23

SERVOGrid Basics - PowerPoint PPT Presentation


  • 78 Views
  • Uploaded on

SERVOGrid Basics. SERVOGrid is our project to build a distributed computing infrastructure to support earthquake simulation codes. We use Web Services for this. We build portal interfaces to these web services out of portlets. Portlets for Java portals are standardized (JSR 168)

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'SERVOGrid Basics' - damia


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
servogrid basics
SERVOGrid Basics
  • SERVOGrid is our project to build a distributed computing infrastructure to support earthquake simulation codes.
    • We use Web Services for this.
    • We build portal interfaces to these web services out of portlets.
      • Portlets for Java portals are standardized (JSR 168)
      • All major commercial vendors, many open source projects support.
      • GEON, NEESGrid, LEAD portal use also.
  • Distributed Computing Infrastructure
    • Services for remotely executing applications, managing distributed files in logical units, monitoring applications, archiving user projects, and so on.
    • Variously called “Grids” (DOE), “Cyberinfrastructure” (NSF), “e-Science” (UK e-Science program), ….
  • The above is sometimes referred to as an “Execution Grid”.
    • We designed this around a web service architecture.
    • See links at the end for more information.
  • But what about the data?
    • Need ways to access remote data sources through programming interfaces.
    • Practice often is to download (via FTP) entire catalogs and hand edit to get data you want in the format you want.
    • We need to automate this.
some servo apps and their data
Some SERVO Apps and Their Data
  • Disloc: handles multiple arbitrarily dipping dislocations (faults) in an elastic half-space.
    • Relies upon geometric fault models.
  • GeoFEST: Three-dimensional viscoelastic finite element model for calculating nodal displacements and tractions. Allows for realistic fault geometry and characteristics, material properties, and body forces.
    • Relies upon fault models with geometric and material properties.
  • VirtualCalifornia: Program to simulate interactions between vertical strike-slip faults using an elastic layer over a viscoelastic half-space.
    • Relies upon fault and fault friction models.
  • RDAHMM: Time series analysis program based on Hidden Markov Modeling. Produces feature vectors and probabilities for transitioning from one class to another.
    • Used to analyze GPS and seismic catalogs.
where is the data
Where Is the Data?
  • QuakeTables Fault Database
    • SERVO’s fault repository for California.
    • Compatible with GeoFEST, Disloc, and VirtualCalifornia
    • http://infogroup.usc.edu:8080/public.html
  • GPS Data sources and formats (RDAHMM and others).
    • JPL: ftp://sideshow.jpl.nasa.gov/pub/mbh
    • SOPAC: ftp://garner.ucsd.edu/pub/timeseries
    • USGS: http://pasadena.wr.usgs.gov/scign/Analysis/plotdata/
  • Seismic Event Data (RDAHMM and others)
    • SCSN: http://www.scec.org/ftp/catalogs/SCSN
    • SCEDC: http://www.scecd.scec.org/ftp/catalogs/SCEC_DC
    • Dinger-Shearer: http://www.scecdc.org/ftp/catalogs/dinger-shearer/dinger-shearer.catalog
    • Haukkson: http://www.scecdc.scec.org/ftp/catalogs/hauksson/Socal
what do we want to do
What Do We Want to Do?
  • Web Services provide a programming interfaces to data so that you can build remote clients.
  • SERVO portal work houses these Web Service clients.
  • Portal also houses the clients for the remote simulation services (RDAHMH, GeoFEST, Potts model codes, etc).
    • As we have seen, these codes work directly with data catalogs
  • Typically the codes must process and reformat the data catalog entries in code-specific ways.
    • Done by hand by most developers, which is fine for development.
    • But not fine for mature codes or automated applications.
  • Currently, our solution is to harvest data from online catalogs and transform into GIS compatible formats in DBs.
    • Discuss a couple of examples
  • But we would like to experiment with building services that are closer to the source.
    • Finer resolution data products
    • Academic exercise in implementing reliable sensor network services and streaming applications.
  • SensorML family from Open Geospatial Consortium provides the standards.
open gis services
Open GIS Services
  • GML abstract data models can encode data but you need services to interact with the remote data.
  • Some example OGC services include
    • Web Feature Service: for retrieving GML encode features, like faults, roads, county boundaries, GPS station locations,….
    • Web Map Service: for creating maps out of Web Features
    • Sensor Grid Services: for working with streaming, time-stamped data.
  • Problems with OGC services
    • Not (yet) Web Service compliant
      • “Pre” web service, no SOAP or WSDL
      • Use instead HTTP GET/POST conventions.
    • Often define general Web Service services as specialized standards
      • Information services
      • Notification services in sensor grids
using gml to encode gps
Using GML to Encode GPS
  • Collected GPS data for each site is made available through online catalogs.
    • Using various text formats.
  • We have developed GML descriptions of GPS using Feature.xsd schema.
  • Full GML GPS schema: http://www.crisisgrid.org/files/gps.xsd.
  • Supports JPL, SOPAC, and USGS formats.
  • Other efforts: Feature encodings of Seismic and Fault catalogs.
    • http://www.crisisgrid.org/html/servo_-_gps.html
anatomy of web feature service
Anatomy of Web Feature Service
  • GML provides the data model, but we must still provide a remote programming interface to the data.
  • Web Feature Service provides three major services as described in OGC specification:
    • GetCapabilities: The clients (WMS servers or users) starts with requesting a document from WFS which describes it’s abilities. When a getCapabilities request arrives, the server dynamically creates a capabilities document and returns this.
      • This is OGC’s formalization of metadata, so important to GEON, ESG, etc.
    • DescribeFeatureType: After the client receives the capabilities document he/she can request a more detailed description for any of the features listed in the WFS capabilities document.
      • The WFS returns an XML schema that describes the requested feature.
      • Metadata about a specific entry.
    • GetFeature: The client can ask the WFS to return a particular portion of any feature data.
      • GetFeature requests contain some property names of the feature and a Filter element to describe the query.
      • The WFS extracts the query and bounding box from the filter and queries the feature databases.
      • The results obtained from the DB query are converted the feature’s GML format and returned to the client as a FeatureCollection.
web map services
Web Map Services
  • OGC Web Map Servers create digital maps from abstract data sets retrieved from Web Feature Services.
    • Layer features are overlayed.
  • WMS servers can be daisy chained.
  • Our research efforts here are to build Web Service version of the WMS as well as portlet clients.
  • We are developing backward compatibility with existing WMS.
    • Purdue CAAGIS for Indiana maps.
    • NASA JPL OnEarth WMS.
  • Future plans are to replace current GMT portal tools with WMS, combine maps with GeoFEST and Virtual California application output.
next steps and questions jan
We are very interested in starting a new Sensor Grid project.

Implement Open Geospatial Consortium standards for Sensor Grids (see supplemental slides).

We are outlining a more ambitious integration of Robert Granat’s RDAHMM with GPS data.

I will be visiting JPL in early February.

Some questions we have are on the right.

Direct (or at least closer) access to GPS stations?

Any documentation on the filtering process?

What kind of filters are applied on the raw data? Can it be automated?

How is data stored, in a DB or just in files?

How do the stations communicate with the servers?

What protocols are being used for transport, etc?

Are there any web-based interfaces to stations or to the data stores?

Are the GPS stations configurable?

How often do they send out data, can you change the interval?

What are the most important data formats for you? (JPL,SOPAC,USGS)

Are there any other formats we need to support?

What (other) kinds of applications are there to use the GPS data?

Next Steps and Questions (Jan)
hpsearch overview
HPSearch Overview
  • HPSearch is a system for scripting and controlling web services and controlling data streams.
    • Uses JavaScript (Rhino) for scripting
    • Uses NaradaBrokering for controlling communications between services.
    • Name is historical
  • Basic premise is that it can be used to hook together streams of data to successive filters (services) that are controlled through a client environment.
basic hpsearch parts
Basic HPSearch Parts
  • Kernel Components
    • Shell: processes the extended JavaScript
    • Service Manager: this is a scheduler for load balancing between several services.
    • RequestHandler: monitors individual remote resources (may include web services but also DBs or plain old URLs).
  • WSProxy facilitates wrapping legacy applications as streaming web services
    • Manages both the control invocations from the shell and the NB-based streaming data connections
    • Comes with two interfaces
      • Runnable: you extend this Java interface for new services.
      • Wrapped: this is a legacy/non-WS service such as a perl or matlab script.
hpsearch architecture

HPSearch Kernel

Data is read / written from / to various data resources

HPSearch Kernel

Request Handler

Files, Sockets,

Topics

Javascript Shell

Request Handler

Service Manager

DataBase

URIHandler

Web Service EP

DBHandler

Web

Service

WSDLHandler

Other Objects

WSProxyHandler

Data Flows as a stream of messages between WSProxies through the brokering network

Broker Network

WSProxy

WSProxy

Service

Service

Stream Proxy

Kernel connects to nearest broker and communicates with other kernels with messages

HPSearch

Kernel

Handlers steer web services via simple SOAP requests

HPSearch Architecture
hpsearch for rdahmm gps

Sensor Network / GPS Database

Actual Data Flow between components takes place through the brokering network

Start Activity,

Web Service

Virtual Data Flow between various components

Narada Broker

Links between Brokers

WSProxy wrapped service

Kernel manages (steers) the various services

HPSearch Kernel

Service distribution

RDAHMM

Data Filter

Matlab Script

HPSearch for RDAHMM + GPS
more information
More Information
  • My email:
    • mpierce@cs.indiana.edu
  • Overview (submitted to ACES special issue of PAGEOPH).
    • http://www.servogrid.org/slide/GEM/SERVO/ISERVO_ACES_PAGEOPH.doc
  • QuakeSim Project Page:
    • http://quakesim.jpl.nasa.gov/
  • QuakeSim Portal:
    • http://www.complexity.ucs.indiana.edu:8282.
  • QuakeTables Fault Database:
    • http://infogroup.usc.edu:8080/public.html
  • Community Grids Lab GIS Grid Development:
    • http://www.crisisgrid.org/.
    • See particularly http://www.crisisgrid.org/html/servo.html for GML information.
    • See http://www.crisisgrid.org/html/wfs.html for more on Web Feature Service.
    • WMS and Information Services information coming soon.
supplemental slides sensor web work

Supplemental Slides: Sensor Web Work

Slides Prepared by Galip Aydin

sensor web
Sensor Web
  • Goal: To create Web-based sensor networks by exploiting web-connected sensors (flood gauges, air pollution monitors etc.).
  • This would make all sensors and repositories of sensor data discoverable, accessible and where applicable controllable via the WWW.
  • OGC describes 5 important encoding and service standards for a Sensor Web Implementation (see next slide).
ogc sensor web enablement
OGC-Sensor Web Enablement
  • SensorML: XML encoding language for sensors. Used to discover, query and control Web-resident sensors.
  • Observations & Measurements: The general models and an XML encoding for what a sensor observes or measures (The value returned by or derived from a sensor observation -e.g. quantity, count, boolean, category, ordered category, position-).
  • Sensor Collection Service: A service to fetch observations from a sensor or constellation of sensors. Provides real time or archived observed values. Clients can also obtain information that describes the associated sensors and platforms.
  • Sensor Planning Service: A service by which a client can determine collection feasibility for a desired set of collection requests for one or more mobile sensors/platforms, or the client may submit collection requests directly to these sensors/platforms. SPS enables sensor tasking, acquisition requests, processing and simulation requests, and registration for alert notification.
  • Web Notification Service: A service by which a client may conduct a dialog with one or more other services. Provides a means for Sensor Planning Services to alert people, software, or other sensor systems of SPS results or alerts regarding phenomena of interest.
sensor web enhancements 1
Sensor Web Enhancements 1
  • We think that with some changes and additions a minimal and high performance Sensor Web with live sensor data capability can be realized.
  • We realize that some of these service specifications are trying to solve problems common to the Web Services community such as notification.
  • There are several Notification specifications for Web Services such as WS-Notification and WS-Eventing. Considering the industrial backing and huge WS community support we can expect these specifications to evolve into standards.
  • Community Grids lab has considerable experience on Notification issues and currently a WS-Notification enabled version of NaradaBrokering is being developed.
sensor web enhancements 2
Sensor Web Enhancements 2
  • Sensor Planning service is designed as an asset management service. For our purposes existence of such a service is not crucial instead an Information Service with search and discovery capabilities is preferable.
  • We are developing a High Performance Fault Tolerant Information Service for GIS applications and with little modifications this service can handle the functionality expected from Sensor Planning Service.
  • The Information Service will provide search and discovery interfaces and WSDL addresses for Sensor Collection Services.
sensor web enhancements 3
Sensor Web Enhancements 3
  • In some cases immediate access to the sensor data is crucial. In such cases we expect to have some sort of a streaming data channel for live sensor data available to the clients.
  • We plan to use Narada Brokering to provide us such a capability.
  • Narada Brokering allows message producers to publish messages to topics and subscribers of these topics immediately receives the published messages.
  • We can create topics according to the types of the sensors and subscribe clients to these topics according to their preferences. Thus Narada Brokering can provide a streaming channel between sensors and the clients.
naradabrokering overview
NaradaBrokering Overview
  • NB is software for routing messages that has been developed by Shrideep Pallickara at CGL for several years.
    • Topic-based publish/subscribe system
  • Implements JMS, JXTA, and its own APIs.
  • Allows you to construct hierarchical broker networks for efficient routing.
  • Can route “anything”
    • HTTP, TCP/IP, UDP
    • Binary data for collaborative shared display (www.anabas.com)
    • GridFTP via proxy
    • SOAP messages
    • WS-Eventing and WS-ReliableMessaging
  • Moving toward supporting SOAP 1.2-style messaging.