The grid open grid services architecture and net
This presentation is the property of its rightful owner.
Sponsored Links
1 / 49

The Grid, Open Grid Services Architecture and .NET PowerPoint PPT Presentation


  • 92 Views
  • Uploaded on
  • Presentation posted in: General

The Grid, Open Grid Services Architecture and .NET. Ian Foster Mathematics and Computer Science Division Argonne National Laboratory and Department of Computer Science The University of Chicago http://www.mcs.anl.gov/~foster. Partial Acknowledgements.

Download Presentation

The Grid, Open Grid Services Architecture and .NET

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The grid open grid services architecture and net

The Grid, Open Grid Services Architectureand .NET

Ian Foster

Mathematics and Computer Science Division

Argonne National Laboratory

and

Department of Computer Science

The University of Chicago

http://www.mcs.anl.gov/~foster


Partial acknowledgements

Partial Acknowledgements

  • Open Grid Services Architecture design

    • Carl Kesselman, Karl Czajkowski @ USC/ISI

    • Steve Tuecke @ANL

    • Jeff Nick, Steve Graham, Jeff Frey @ IBM

  • Grid services collaborators at ANL

    • Kate Keahey, Gregor von Laszewski

    • Thomas Sandholm, Jarek Gawor, John Bresnahan

  • Globus Toolkit R&D also involves many fine scientists & engineers at ANL, USC/ISI, and elsewhere (see www.globus.org)

  • Strong links with many EU, UK, US Grid projects

  • Support from DOE, NASA, NSF, IBM, Microsoft


Goals

Goals

  • Communicate the purpose, significance, state, adoption, & future of Grid technology

  • Persuade you that Grid technology represents an opportunity & challenge, for

    • Developers: Grids aren’t only (or particularly) about science or servers—themes of virtualization, service discovery, service management, and QoS delivery are universal

    • Researchers & educators: Rapid uptake in industry & science represents an exceptional opportunity for impact


Overview

Overview

  • Origins: Resource sharing within scientific collaborations

    • Science drivers & science Grid projects

    • Globus Toolkit

  • Evolution: Resource virtualization

    • Commercial drivers

    • OGSA: Grid meets Web services

  • Futures

    • OGSA, GXA, and .NET


Overview1

Overview

  • Origins: Resource sharing within scientific collaborations

    • Science drivers & science Grid projects

    • Globus Toolkit

  • Evolution: Resource virtualization

    • Commercial drivers

    • OGSA: Grid meets Web services

  • Futures

    • OGSA, GXA, and .NET


E science the original grid driver

E-Science: The Original Grid Driver

  • Pre-electronic science

    • Theorize &/or experiment, in small teams

  • Post-electronic science

    • Construct and mine very large databases

    • Develop computer simulations & analyses

    • Access specialized devices remotely

    • Exchange information within distributed multidisciplinary teams

    • Need to manage dynamic, distributed infrastructures, services, and applications


Escience application sloan digital sky survey analysis

Galaxy cluster

size distribution

Chimera Virtual Data System

+ GriPhyN Virtual Data Toolkit

+ iVDGL Data Grid (many CPUs)

eScience Application:Sloan Digital Sky Survey Analysis

Size distribution of

galaxy clusters?

www.griphyn.org/chimera


Nasa s information power grid aviation safety

Human Models

NASA’s Information Power Grid: Aviation Safety

Wing Models

  • Lift Capabilities

  • Drag Capabilities

  • Responsiveness

Stabilizer Models

Airframe Models

  • Deflection capabilities

  • Responsiveness

Crew Capabilities

- accuracy

- perception

- stamina

- re-action times

- SOPs

Engine Models

  • Braking performance

  • Steering capabilities

  • Traction

  • Dampening capabilities

  • Thrust performance

  • Reverse Thrust performance

  • Responsiveness

  • Fuel Consumption

Landing Gear Models


Life sciences telemicroscopy

Life Sciences: Telemicroscopy

DATA ACQUISITION

PROCESSING,ANALYSIS

ADVANCEDVISUALIZATION

NETWORK

COMPUTATIONALRESOURCES

IMAGING

INSTRUMENTS

LARGE DATABASES


And thus the grid

And Thus: The Grid

“Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations”


Underlying technical requirements

Underlying Technical Requirements

  • Dynamic formation and management of virtual organizations

  • Online negotiation of access to services: who, what, why, when, how

  • Configuration of applications and systems able to deliver multiple qualities of service

  • Autonomic management of distributed infrastructures, services, and applications


The grid world current status

The Grid World: Current Status

  • Dozens of major Grid projects in scientific & technical computing/research & education

  • Open source Globus Toolkit™ a de facto standard for major protocols & services

    • Simple protocols & APIs for authentication, discovery, access, etc.

    • Information-centric design

    • Large user and developer base

    • Multiple commercial support providers

    • Enabler of numerous tools and applications

  • Global Grid Forum: community & standards


Globus toolkit v2 on win32 thanks microsoft

Globus Toolkit v2 on Win32(Thanks, Microsoft)

  • Goal: Enable port of Grid clients to Windows

    • E.g. Condor-G, NetSolve, custom apps, etc.

  • Port of GT2 “clients” to Win32 includes

    • GT2 infrastructure: common, security, io

    • Job submission: GRAM client

    • Data transfer: GridFTP client, GASS client/server

  • Not porting GT2 servers: GRAM, GridFTP, MDS

  • GT2 core in alpha now, others in progress

  • Future work directed towards OGSA (see below)


Overview2

Overview

  • Origins: Resource sharing within scientific collaborations

    • Science drivers & science Grid projects

    • Globus Toolkit

  • Evolution: Resource virtualization

    • Commercial drivers

    • OGSA: Grid meets Web services

  • Futures

    • OGSA, GXA, and .NET


Resource sharing within vos is not unique to science

Resource Sharing within “VOs” is Not Unique to Science!

  • Fragmentation of enterprise infrastructure

    • Driven by cheap servers, fast nets, ubiquitous Internet, eBusiness workloads

  • B2B, B2C, C2C interactions

    • Collaborative computing, P2P, …

  • Also fundamental to virtualization

    • Emerging service infrastructure, utility computing models, economies of scale

    • Services dynamically instantiated across device spectrum


Virtualization fundamental to future computing infrastructure

Resource &

service

aggregation

Delivery of virtualized services with QoS

guarantees

Dynamic, secure

service discovery

& composition

Virtualization Fundamental toFuture Computing Infrastructure

Larger, more integrated

More connected

Dynamically provisioned

Less capable, integrated

Less connected

User service locus

Device Continuum


Realizing the promise requires significant innovation

Realizing the PromiseRequires Significant Innovation

  • Automation of infrastructure operation to achieve economies of scale

  • Management and component models for service discovery, composition, provisioning

  • New applications and tools powered by distributed services and resources

  • Business and service models to support specialization of function


Grid evolution open grid services architecture

Grid Evolution:Open Grid Services Architecture

  • Refactor Globus protocol suite to enable common base and expose key capabilities

  • Service orientation to virtualize resources and unify resources/services/information

  • Embrace key Web services technologies: WSDL as IDL, leverage commercial efforts

  • Result: standard interfaces & behaviors for distributed system management


Ogsa system structure

OGSA System Structure

  • A standard substrate: the Grid service

    • Standard interfaces and behaviors that address key distributed system issues

    • The “Grid Service Specification”

  • … supports standard service specifications

    • Resource management, databases, workflow, security, diagnostics, etc., etc.

    • Target of current & planned GGF efforts

  • … and arbitrary application-specific services based on these & other definitions


Web services

“Web Services”

  • Increasingly popular standards-based framework for accessing network applications

    • W3C standardization; Microsoft, IBM, Sun, others

  • WSDL: Web Services Description Language

    • Interface Definition Language for Web services

  • SOAP: Simple Object Access Protocol

    • XML-based RPC protocol; common WSDL target

  • WS-Inspection

    • Conventions for locating service descriptions

  • UDDI: Universal Desc., Discovery, & Integration

    • Directory for Web services


Web services example database service

Web Services Example:Database Service

  • WSDL definition for “DBaccess” porttype defines operations and bindings, e.g.:

    • Query(QueryLanguage, Query, Result)

    • SOAP protocol

  • Client C, Java, Python, etc., APIs can then be generated

DBaccess


Transient service instances

Transient Service Instances

  • “Web services” address discovery & invocation of persistent services

    • Interface to persistent state of entire enterprise

  • In Grids, must also support transient service instances, created/destroyed dynamically

    • Interfaces to the states of distributed activities

    • E.g. workflow, video conf., dist. data analysis

  • Significant implications for how services are managed, named, discovered, and used


Ogsi ogsa and web services

OGSI, OGSA, and Web Services

  • OGSI (I = Infrastructure)

    • Small extensions to WSDL

      • Nested serviceType & serviceDataDescription

    • Conventions for naming service instances

      • Handles and references

    • portTypes for common behavior

      • Instance creation, lifetime management, introspection and monitoring, registration, notification, …

  • OGSA (A = Architecture) built on OGSI

    • A collection of Grid service interfaces

      • Resource description & provisioning

      • Higher-level services: messaging services, logging, etc.


The grid service interfaces behaviors service data

GridService

(required)

… other interfaces …

(optional)

Service data access

Explicit destruction

Soft-state lifetime

Standard:

- Notification

- Authorization

- Service creation

- Service registry

- Manageability

- Concurrency

+ application-specific interfaces

The Grid Service =Interfaces/Behaviors + Service Data

Service

data

element

Service

data

element

Service

data

element

Binding properties:

- Reliable invocation

- Authentication

Implementation

Hosting environment/runtime

(“C”, J2EE, .NET, …)


Service data

Service Data

  • A Grid service instance maintains a set of service data elements

    • Described in WSDL extension

    • XML element encapsulated in standard container: name, type, lifetime, etc.

    • Includes basic introspection information, interface-specific data, and application state

  • Pull and push models for information query

    • GridService::FindServiceData operation

      • Pull: queries this information via extensible query language

    • NotificationSource::SubscribeServiceData

      • Push: Subscribe to notification of changes to information


Notification interfaces

Notification Interfaces

  • NotificationSource for client subscription

    • Subscription expression describes which service data element changes are of interest

    • Creates a subscription manager service

      • Manages the lifetime and properties of subscription

  • NotificationSink for asynchronous delivery of notification messages

  • Simple, flexible base with wide variety of uses

    • Dynamic discovery/registry services, monitoring, application error notification, etc.

    • Intermediaries: filter, aggregate, archive, et.c

    • Can integrate commercial messaging services


Grid service example database service

Grid Service Example:Database Service

  • A DBaccess Grid service will support at least two portTypes

    • GridService

    • DBaccess

  • Each has service data

    • GridService: basic introspection information, lifetime, …

    • DBaccess: database type, query languages supported, current load, …, …

  • Maybe other portTypes as well

    • E.g., NotificationSource

Grid

Service

DBaccess

Name, lifetime, etc.

DB info


Lifetime management

Lifetime Management

  • GS instances created by factory or manually; destroyed explicitly or via soft state

    • Negotiation of initial lifetime with a factory (=service supporting Factory interface)

  • GridService interface supports

    • Destroy operation for explicit destruction

    • SetTerminationTime operation for keepalive

  • Soft state lifetime management avoids

    • Explicit client teardown of complex state

    • Resource “leaks” in hosting environments


Factory

Factory

  • Factory interface’s CreateService operation creates a new Grid service instance

    • Reliable creation (once-and-only-once)

  • CreateService operation can be extended to accept service-specific creation parameters

  • Returns a Grid Service Handle (GSH)

    • A globally unique URL, resolves to GSR

    • Uniquely identifies the instance for all time

    • Based on name of a handle resolver

  • Or Grid Service Reference (GSR)


Example transient database services

“Create a database service”

“What services can you create?”

Grid

Service

DBaccess

Name, lifetime, etc.

DB info

“What database services exist?”

Grid

Service

DBaccess

Name, lifetime, etc.

DB info

Example:Transient Database Services

Grid

Service

DBaccess

Factory

Instance name, etc.

Factory info

Grid

Service

Registry

Instance name, etc.

Registry info


Data mining for bioinformatics

Data Mining for Bioinformatics

Community

Registry

Mining

Factory

Database

Service

BioDB 1

Compute Service Provider

User

Application

.

.

.

.

.

.

“I want to create

a personal database

containing data on

e.coli metabolism”

Database

Service

Database

Factory

BioDB n

Storage Service Provider


Data mining for bioinformatics1

Data Mining for Bioinformatics

“Find me a data mining service, and somewhere to store data”

Community

Registry

Mining

Factory

Database

Service

BioDB 1

Compute Service Provider

User

Application

.

.

.

.

.

.

Database

Service

Database

Factory

BioDB n

Storage Service Provider


Data mining for bioinformatics2

Data Mining for Bioinformatics

Community

Registry

Mining

Factory

Database

Service

Handles for Mining

and Database

factories

BioDB 1

Compute Service Provider

User

Application

.

.

.

.

.

.

Database

Service

Database

Factory

BioDB n

Storage Service Provider


Data mining for bioinformatics3

Data Mining for Bioinformatics

Community

Registry

Mining

Factory

Database

Service

“Create a data mining service with initial lifetime 10”

BioDB 1

Compute Service Provider

User

Application

.

.

.

.

.

.

“Create a

database with initial lifetime 1000”

Database

Service

Database

Factory

BioDB n

Storage Service Provider


Data mining for bioinformatics4

Data Mining for Bioinformatics

Community

Registry

Mining

Factory

Database

Service

“Create a data mining service with initial lifetime 10”

BioDB 1

Miner

Compute Service Provider

User

Application

.

.

.

.

.

.

“Create a

database with initial lifetime 1000”

Database

Service

Database

Factory

BioDB n

Database

Storage Service Provider


Data mining for bioinformatics5

Data Mining for Bioinformatics

Community

Registry

Mining

Factory

Database

Service

Query

BioDB 1

Miner

Compute Service Provider

User

Application

.

.

.

.

.

.

Query

Database

Service

Database

Factory

BioDB n

Database

Storage Service Provider


Data mining for bioinformatics6

Data Mining for Bioinformatics

Community

Registry

Mining

Factory

Database

Service

Query

BioDB 1

Miner

Keepalive

Compute Service Provider

User

Application

.

.

.

.

.

.

Query

Database

Service

Database

Factory

Keepalive

BioDB n

Database

Storage Service Provider


Data mining for bioinformatics7

Data Mining for Bioinformatics

Community

Registry

Mining

Factory

Database

Service

BioDB 1

Miner

Keepalive

Compute Service Provider

User

Application

.

.

.

.

.

.

Results

Database

Service

Database

Factory

Keepalive

Results

BioDB n

Database

Storage Service Provider


Data mining for bioinformatics8

Data Mining for Bioinformatics

Community

Registry

Mining

Factory

Database

Service

BioDB 1

Miner

Compute Service Provider

User

Application

.

.

.

.

.

.

Database

Service

Database

Factory

Keepalive

BioDB n

Database

Storage Service Provider


Data mining for bioinformatics9

Data Mining for Bioinformatics

Community

Registry

Mining

Factory

Database

Service

BioDB 1

Compute Service Provider

User

Application

.

.

.

.

.

.

Database

Service

Database

Factory

Keepalive

BioDB n

Database

Storage Service Provider


Ogsa design implementation

OGSA Design & Implementation

  • OGSI (I=Infrastructure) WG in GGF defining core Grid service specification

    • (At least) three implementation efforts

  • Globus Toolkit => GT3 (alpha end 2002)

    • GT3 Core: Grid service specification

    • GT3 Base: Globus Toolkit behaviors

    • CIM resource model, GRAM-2 SLA negotiation, database services, …

  • Other GGF WGs address OGSA security, OGSA-compliant database services, etc.


Overview3

Overview

  • Origins: Resource sharing within scientific collaborations

    • Science drivers & science Grid projects

    • Globus Toolkit

  • Evolution: Resource virtualization

    • Commercial drivers

    • OGSA: Grid meets Web services

  • Futures

    • OGSA, GXA, and .NET


Ogsa implementation

1) OGSA builds on infrastructure

  • Plumbing: WSDL, WS-Security, WS-Routing/Referral, reliable messaging, transactions, etc.

  • Hosting environments

OGSI/OGSA Interfaces

service description,

service provisioning, …

Standard

OGSI

container

Web

services

various

2) to enable virtualization via

  • Service description

  • Service provisioning

Hosting Environment

Resource virtualization

and QoS support

3) Standard container avoids implementing OGSI features in every service instance

OGSA Implementation

OGSI/OGSA Interfaces

service description,

service provisioning, …

Standard

OGSI

container

Web

services

various

Hosting Environment

Resource virtualization

and QoS support


Ogsi gxa and net

OGSI, GXA, and .NET

  • Infrastructure

    • GXA Web services facilities provide OGSI plumbing

    • .NET provides nice hosting environment for Grid services

  • .NET also a good basis for standard OGSI container

  • OGSA can, in turn, provide resource description and provisioning for GXA/.NET

OGSI/OGSA Interfaces

service description,

service provisioning, …

Standard

OGSI

container

Web

services

various

GXA

.NET

Hosting Environment

Resource virtualization

and QoS support

.NET


Building an ogsi container

Building an OGSI Container

  • Service data mgmt, query, subscription

    • Container should provide simple interface for interacting with an instance’s implementation to get and manage dynamic service data

      • Service instance = CLR object

    • Container should handle query processing

      • .NET support for XPath & Xquery allows for rich functionality

    • Container manages notification subscriptions, and drives asynchronous notification messages

  • Soft-state lifetime management

  • Soft-state registration


Other uses of net for ogsa

Other Uses of .NET for OGSA

  • Resource description: WMI

    • WMI provides CIM-based infrastructure

  • Resource provisioning: various

    • Rich Windows support for QoS delivery

      • Real-time CPU scheduling

      • Network QoS (RSVP, diffserv, etc.)

      • Storage quotas?

  • Notification: MSMQ

    • Integrate with MSMQ for notification w/ QoS

  • Extensible factories: CLR


Recap goals

Recap: Goals

  • Communicate the purpose, significance, state, adoption, & future of Grid technology

  • Persuade you that Grid technology represents an opportunity & challenge

    • Developers: Grids aren’t only (or particularly) about science and servers—themes of virtualization, service discovery, service management, and QoS delivery are universal

    • Researchers & educators: Rapid uptake in industry & science represents an exceptional opportunity for impact


For more information

For More Information

  • The Globus Project™

    • www.globus.org

  • Context & research articles

    • www.mcs.anl.gov/~foster

  • Open Grid Services Architecture

    • www.globus.org/ogsa

  • Global Grid Forum

    • www.gridforum.org

    • Edinburgh, July 22-24

    • Chicago, Oct 15-17


  • Login