1 / 31

July 13 2004 Fourth ACES APEC Cooperation for Earthquake Simulation Workshop Beijing China

SERVO Grid: Solid Earth Research Virtual Observatory Grid/Web Services and Portals Supporting Earthquake Science. July 13 2004 Fourth ACES APEC Cooperation for Earthquake Simulation Workshop Beijing China Geoffrey Fox, Marlon Pierce Community Grids Lab, Pervasive Technologies Laboratories

felixcunningham
Download Presentation

July 13 2004 Fourth ACES APEC Cooperation for Earthquake Simulation Workshop Beijing China

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SERVO Grid:Solid Earth Research Virtual ObservatoryGrid/Web Services and Portals Supporting Earthquake Science July 13 2004 Fourth ACES APEC Cooperation for Earthquake Simulation Workshop Beijing China Geoffrey Fox, Marlon Pierce Community Grids Lab, Pervasive Technologies Laboratories Indiana University

  2. What is a Grid? • You won’t find a clear description of what is Grid and how does differ from a collection of programs on the Internet • Grids were once defined as “Internet Scale Distributed Computing” but this isn’t very good as Grids depend as much if not more on data as well as simulations • So I will use: Grids are “Internet Scale Distributed Services” and represent a way of collecting services together in same way that a program (package) collects methods and objects together on a sequential machine. • In this view, Grids are naturally and critically tied to Web Services and are built on top of Web service standards • The high performance computing and e-Science origin of Grids does give some special challenges • Streams, State, Dataflow, High Performance data transfer, link to parallel computers, cross administrative domain access • Grids are built with Web Services and so a Grid Service is a Web Service and the differences are quite small and in particular irrelevant for current version of SERVO

  3. SERVO Strategy • Use “Pure”Web Services which can be augmented when debate between Microsoft/IBM on WSRF/WS-GAF • Note Web Services still evolving • >60 proposed Web Service specifications competing to be “standards” • Only 4.5 out of 60 have made it to Industry WS-I interoperability suite • But essential idea of services and their interconnection with messages will not change!

  4. Some Grid Controversies • 1) There are several proposals for the Web Service extensions needed for Grids – why do we ignore? • OGSI (GT3) • WSRF (GT4) • WS-GAF (Newcastle) • WS-I+ (Pure Web Services) • We use WS-I+ approach – can later add extensions when consensus clear • This approach adopted by next phase of UK e-Science Program • 2) Web Services are too slow as use HTTP with clumsy ASCII XML data (SOAP)? • Negotiate with “slow clumsy” but totally interoperable messages • Agree on very fast data stream protocol and encoding • i.e. separate control channel from data channel

  5. SERVOGrid Requirements • Seamless Access to Data repositories and large scale computers • Integration of multiple data sources including sensors, databases, file systems with analysis system • Including filtered OGSA-DAI (Grid database access) • Rich meta-data generation and access with SERVOGrid specific Schema extending openGIS (Geography as a Web service) standards and using Semantic Grid • Portalswith component model for user interfaces and web control of all capabilities • Collaboration to support world-wide work • Basic Grid tools: workflow and notification • Notmetacomputing

  6. iSERVO Strategy • Agree on what (type of) resources and capabilities need to put on the ISERVO Grid • Computers, instruments, databases, visualization, maps, job submittal …. • Agree on interfaces to resources from OGSA-DAI (databases) to particular data structures (GML/OpenGIS) – specify in XML • Implement Resources and Capabilities as Services • User Interface should be a portlet that can be integrated by the portal into web interface • Make certain overarching Grid capabilities such as workflow, federation and metadata are sufficient • SERVO Grid is a prototype of this strategy using several US sites rather than several countries • Can be naturally extended to iSERVO, education, emergency response by extending resources • Web Service Architecture ensures continued interoperability and extensibility

  7. (i)SERVO Web (Grid) Services • Programs:All applications wrapped as Services using proxy strategy • Job Submission: supports remote batch and shell invocations • Used to execute simulation codes (VC suite, GeoFEST, etc.), mesh generation (Akira/Apollo) and visualization packages (RIVA, GMT). • File management: • Uploading, downloading, backend crossloading (i.e. move files between remote servers) • Remote copies, renames, etc. • Job monitoring • Workflow: Apache Ant-based remote service orchestration (NCSA) • Move towards a BPEL framework (can still implement with ANT) • Database services: support SQL queries • Expect Simpler version of OGSA-DAI (“Web Service-DAI”) Grid Database • Data services: support interactions with XML-based fault and surface observation data. • For simulation generated faults (i.e. from Simplex) • XML data model being adopted for common formats with translation services to “legacy” formats. • Migrating to Geography Markup Language (GML) descriptions.

  8. SERVOGrid Application Descriptions • Codes range from simple “rough estimate” codes to parallel, high performance applications. • Disloc: handles multiple arbitrarily dipping dislocations (faults) in an elastic half-space. • Simplex: inverts surface geodetic displacements for fault parameters using simulated annealing downhill residual minimization. • GeoFEST: Three-dimensional viscoelastic finite element model for calculating nodal displacements and tractions. Allows for realistic fault geometry and characteristics, material properties, and body forces. • VirtualCalifornia: Program to simulate interactions between vertical strike-slip faults using an elastic layer over a viscoelastic half-space • RDAHMM: Time series analysis program based on Hidden Markov Modeling. Produces feature vectors and probabilities for transitioning from one class to another. • PARK: Boundary element program to calculate fault slip velocity history based on fault frictional properties.a model for unstable slip on a single earthquake fault. • Preprocessors, mesh generators • Visualization tools: RIVA, GMT

  9. SERVOGrid Codes, Relationships Elastic Dislocation Inversion Viscoelastic FEM Viscoelastic Layered BEM Elastic Dislocation Pattern Recognizers Fault Model BEM This linkage called Workflow in Grid/Web Service parlance

  10. Service-1 Service-3 Role of Workflow • Programming the Grid: Workflow describes linkage between services • As distributed, linkage must be by messages • Linkage is two-way and has both control and data • Apply to multi-scale (complexity) linkage, multi-program linkage, link visualization to simulation, GIS to simulations and viz filters to each other • Microsoft-IBM specification BPEL is current preferred Web Service XML specification of workflow Service-2

  11. Security: Authentication and Authorization • Authentication describes who the user is • Authorization describes what a given user can do • What data and computers can be accessed • Basically a database • Current portal uses password accounts and provides services for free for demonstration. • iSERVO should decide on “charging for” services • We have (through Community portal effort OGCE) support for GSI and Kerberos authentication services. • These just plug in and replace the default login service. • Authorization is currently simple: you can only reach your files. • iSERVO should develop an authorization policy • Simultaneous Cross Administrative Domain access is a very hard Grid problem and no consensus as to good solution • Systematic use of Services helps security/privacy/IP issues as “danger of misuse” is lower for services (which have limited privileges) than for direct computer access

  12. Each Service has its own portlet Individual portlet for the Proxy Manager Use tabs or choose different portlets to navigate through interfaces to different services 2 Other Portlets

  13. Important Lessons/Principles • Use OGCE Portal Architecture and portal services • Can expect GGF activities like OGSA to define/refine interfaces and projects around the world to produce more powerful services • Obsolescing of implementations is a consequence of interoperability • Use Grids of Grids of Services Architecture • Interoperable Component Grids Built from interoperable services • Collaboration, Compute, Database, GIS, Sensor, Visualization Grids • Build a GIS (Geographical Information Systems) Grid spanning simulation/crisis management and different fields with openGIS compliance • openGIS has defined Web Service Interfaces • Visualization should build on these • Geoscience Education Grid by transformations on research grid • Emergency Response and Planning Grids by adding real-time control/collaboration and GIS tools • These additions common to all crises • Collaboration between Beihang University and Indiana University to produce Web Service based audio/video conferencing

  14. Electricity CIGrid Security Notification Workflow Messaging Flood CIGrid … … Earthquake CIGrid Flood Servicesand Filters Earthquake Services Portals Collaboration Grid Visualization Grid Sensor Grid GIS Grid Compute Grid Data Access/Storage Registry Metadata Core Grid Services Physical Network Critical Infrastructure (CI) Grids built as Grids of Grids of Services

  15. Field Trip Data ? GISGrid Discovery Services RepositoriesFederated Databases Streaming Data Sensors Database Database Sensor Grid Database Grid Research Education SERVO Grid Compute Grid Customization Services From Researchto Education Data FilterServices ResearchSimulations Analysis and VisualizationPortal EducationGrid Computer Farm Earthquake Research and Education Grids

  16. Further iSERVO Challenges • Make everything a Service • Think about Data Curation • Set up policies for observational data and criteria for inclusion in iSERVO data repositories • Think about Data Provenance • Generate and maintain metadata describing ownership, origins and transformations • Applies to both “experimental data” and results from simulations (visualizations) • Curation and Provenance change in research methodologies and requires funding! • Education and Emergency Response/Planning interesting offshoots of iSERVO

  17. QuakeSim Portal Shots

  18. Session Archive Service

  19. A Restored Session

  20. Modify Stored Session Values

  21. Geometry and Mesh Plotting Service(and the Mesh Generation Service)

  22. Interactive Job Submission

  23. Archive and File Download Services

  24. Non-Interactive Job Submission

  25. Job Monitoring Service

  26. Some IDL Plots of GeoFEST Output

  27. Some GMT Services and Plots

  28. iSERVO Example: Finley • Finley is a finite element code being developed by the QUAKES group at the University of Queensland. • Compatible with GeoFEST-style geometry models and mesh generation tools. • So we can reuse the services we wrapped for GeoFEST. • The Finley application itself is a separate service and also has a separate visualization service.

  29. Setting Up Finley Simulation of Northridge Selected Fault Components Select Fault from USC database

  30. Run Finley, Retrieve Generate Movie

More Related