1 / 49

The Grid, Open Grid Services Architecture and .NET

The Grid, Open Grid Services Architecture and .NET. Ian Foster Mathematics and Computer Science Division Argonne National Laboratory and Department of Computer Science The University of Chicago http://www.mcs.anl.gov/~foster. Partial Acknowledgements.

akeem-pitts
Download Presentation

The Grid, Open Grid Services Architecture and .NET

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. The Grid, Open Grid Services Architectureand .NET Ian Foster Mathematics and Computer Science Division Argonne National Laboratory and Department of Computer Science The University of Chicago http://www.mcs.anl.gov/~foster

  2. Partial Acknowledgements • Open Grid Services Architecture design • Carl Kesselman, Karl Czajkowski @ USC/ISI • Steve Tuecke @ANL • Jeff Nick, Steve Graham, Jeff Frey @ IBM • Grid services collaborators at ANL • Kate Keahey, Gregor von Laszewski • Thomas Sandholm, Jarek Gawor, John Bresnahan • Globus Toolkit R&D also involves many fine scientists & engineers at ANL, USC/ISI, and elsewhere (see www.globus.org) • Strong links with many EU, UK, US Grid projects • Support from DOE, NASA, NSF, IBM, Microsoft

  3. Goals • Communicate the purpose, significance, state, adoption, & future of Grid technology • Persuade you that Grid technology represents an opportunity & challenge, for • Developers: Grids aren’t only (or particularly) about science or servers—themes of virtualization, service discovery, service management, and QoS delivery are universal • Researchers & educators: Rapid uptake in industry & science represents an exceptional opportunity for impact

  4. Overview • Origins: Resource sharing within scientific collaborations • Science drivers & science Grid projects • Globus Toolkit • Evolution: Resource virtualization • Commercial drivers • OGSA: Grid meets Web services • Futures • OGSA, GXA, and .NET

  5. Overview • Origins: Resource sharing within scientific collaborations • Science drivers & science Grid projects • Globus Toolkit • Evolution: Resource virtualization • Commercial drivers • OGSA: Grid meets Web services • Futures • OGSA, GXA, and .NET

  6. E-Science: The Original Grid Driver • Pre-electronic science • Theorize &/or experiment, in small teams • Post-electronic science • Construct and mine very large databases • Develop computer simulations & analyses • Access specialized devices remotely • Exchange information within distributed multidisciplinary teams • Need to manage dynamic, distributed infrastructures, services, and applications

  7. Galaxy cluster size distribution Chimera Virtual Data System + GriPhyN Virtual Data Toolkit + iVDGL Data Grid (many CPUs) eScience Application:Sloan Digital Sky Survey Analysis Size distribution of galaxy clusters? www.griphyn.org/chimera

  8. Human Models NASA’s Information Power Grid: Aviation Safety Wing Models • Lift Capabilities • Drag Capabilities • Responsiveness Stabilizer Models Airframe Models • Deflection capabilities • Responsiveness Crew Capabilities - accuracy - perception - stamina - re-action times - SOPs Engine Models • Braking performance • Steering capabilities • Traction • Dampening capabilities • Thrust performance • Reverse Thrust performance • Responsiveness • Fuel Consumption Landing Gear Models

  9. Life Sciences: Telemicroscopy DATA ACQUISITION PROCESSING,ANALYSIS ADVANCEDVISUALIZATION NETWORK COMPUTATIONALRESOURCES IMAGING INSTRUMENTS LARGE DATABASES

  10. And Thus: The Grid “Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations”

  11. Underlying Technical Requirements • Dynamic formation and management of virtual organizations • Online negotiation of access to services: who, what, why, when, how • Configuration of applications and systems able to deliver multiple qualities of service • Autonomic management of distributed infrastructures, services, and applications

  12. The Grid World: Current Status • Dozens of major Grid projects in scientific & technical computing/research & education • Open source Globus Toolkit™ a de facto standard for major protocols & services • Simple protocols & APIs for authentication, discovery, access, etc. • Information-centric design • Large user and developer base • Multiple commercial support providers • Enabler of numerous tools and applications • Global Grid Forum: community & standards

  13. Globus Toolkit v2 on Win32(Thanks, Microsoft) • Goal: Enable port of Grid clients to Windows • E.g. Condor-G, NetSolve, custom apps, etc. • Port of GT2 “clients” to Win32 includes • GT2 infrastructure: common, security, io • Job submission: GRAM client • Data transfer: GridFTP client, GASS client/server • Not porting GT2 servers: GRAM, GridFTP, MDS • GT2 core in alpha now, others in progress • Future work directed towards OGSA (see below)

  14. Overview • Origins: Resource sharing within scientific collaborations • Science drivers & science Grid projects • Globus Toolkit • Evolution: Resource virtualization • Commercial drivers • OGSA: Grid meets Web services • Futures • OGSA, GXA, and .NET

  15. Resource Sharing within “VOs” is Not Unique to Science! • Fragmentation of enterprise infrastructure • Driven by cheap servers, fast nets, ubiquitous Internet, eBusiness workloads • B2B, B2C, C2C interactions • Collaborative computing, P2P, … • Also fundamental to virtualization • Emerging service infrastructure, utility computing models, economies of scale • Services dynamically instantiated across device spectrum

  16. Resource & service aggregation Delivery of virtualized services with QoS guarantees Dynamic, secure service discovery & composition Virtualization Fundamental toFuture Computing Infrastructure Larger, more integrated More connected Dynamically provisioned Less capable, integrated Less connected User service locus Device Continuum

  17. Realizing the PromiseRequires Significant Innovation • Automation of infrastructure operation to achieve economies of scale • Management and component models for service discovery, composition, provisioning • New applications and tools powered by distributed services and resources • Business and service models to support specialization of function

  18. Grid Evolution:Open Grid Services Architecture • Refactor Globus protocol suite to enable common base and expose key capabilities • Service orientation to virtualize resources and unify resources/services/information • Embrace key Web services technologies: WSDL as IDL, leverage commercial efforts • Result: standard interfaces & behaviors for distributed system management

  19. OGSA System Structure • A standard substrate: the Grid service • Standard interfaces and behaviors that address key distributed system issues • The “Grid Service Specification” • … supports standard service specifications • Resource management, databases, workflow, security, diagnostics, etc., etc. • Target of current & planned GGF efforts • … and arbitrary application-specific services based on these & other definitions

  20. “Web Services” • Increasingly popular standards-based framework for accessing network applications • W3C standardization; Microsoft, IBM, Sun, others • WSDL: Web Services Description Language • Interface Definition Language for Web services • SOAP: Simple Object Access Protocol • XML-based RPC protocol; common WSDL target • WS-Inspection • Conventions for locating service descriptions • UDDI: Universal Desc., Discovery, & Integration • Directory for Web services

  21. Web Services Example:Database Service • WSDL definition for “DBaccess” porttype defines operations and bindings, e.g.: • Query(QueryLanguage, Query, Result) • SOAP protocol • Client C, Java, Python, etc., APIs can then be generated DBaccess

  22. Transient Service Instances • “Web services” address discovery & invocation of persistent services • Interface to persistent state of entire enterprise • In Grids, must also support transient service instances, created/destroyed dynamically • Interfaces to the states of distributed activities • E.g. workflow, video conf., dist. data analysis • Significant implications for how services are managed, named, discovered, and used

  23. OGSI, OGSA, and Web Services • OGSI (I = Infrastructure) • Small extensions to WSDL • Nested serviceType & serviceDataDescription • Conventions for naming service instances • Handles and references • portTypes for common behavior • Instance creation, lifetime management, introspection and monitoring, registration, notification, … • OGSA (A = Architecture) built on OGSI • A collection of Grid service interfaces • Resource description & provisioning • Higher-level services: messaging services, logging, etc.

  24. GridService (required) … other interfaces … (optional) Service data access Explicit destruction Soft-state lifetime Standard: - Notification - Authorization - Service creation - Service registry - Manageability - Concurrency + application-specific interfaces The Grid Service =Interfaces/Behaviors + Service Data Service data element Service data element Service data element Binding properties: - Reliable invocation - Authentication Implementation Hosting environment/runtime (“C”, J2EE, .NET, …)

  25. Service Data • A Grid service instance maintains a set of service data elements • Described in WSDL extension • XML element encapsulated in standard container: name, type, lifetime, etc. • Includes basic introspection information, interface-specific data, and application state • Pull and push models for information query • GridService::FindServiceData operation • Pull: queries this information via extensible query language • NotificationSource::SubscribeServiceData • Push: Subscribe to notification of changes to information

  26. Notification Interfaces • NotificationSource for client subscription • Subscription expression describes which service data element changes are of interest • Creates a subscription manager service • Manages the lifetime and properties of subscription • NotificationSink for asynchronous delivery of notification messages • Simple, flexible base with wide variety of uses • Dynamic discovery/registry services, monitoring, application error notification, etc. • Intermediaries: filter, aggregate, archive, et.c • Can integrate commercial messaging services

  27. Grid Service Example:Database Service • A DBaccess Grid service will support at least two portTypes • GridService • DBaccess • Each has service data • GridService: basic introspection information, lifetime, … • DBaccess: database type, query languages supported, current load, …, … • Maybe other portTypes as well • E.g., NotificationSource Grid Service DBaccess Name, lifetime, etc. DB info

  28. Lifetime Management • GS instances created by factory or manually; destroyed explicitly or via soft state • Negotiation of initial lifetime with a factory (=service supporting Factory interface) • GridService interface supports • Destroy operation for explicit destruction • SetTerminationTime operation for keepalive • Soft state lifetime management avoids • Explicit client teardown of complex state • Resource “leaks” in hosting environments

  29. Factory • Factory interface’s CreateService operation creates a new Grid service instance • Reliable creation (once-and-only-once) • CreateService operation can be extended to accept service-specific creation parameters • Returns a Grid Service Handle (GSH) • A globally unique URL, resolves to GSR • Uniquely identifies the instance for all time • Based on name of a handle resolver • Or Grid Service Reference (GSR)

  30. “Create a database service” “What services can you create?” Grid Service DBaccess Name, lifetime, etc. DB info “What database services exist?” Grid Service DBaccess Name, lifetime, etc. DB info Example:Transient Database Services Grid Service DBaccess Factory Instance name, etc. Factory info Grid Service Registry Instance name, etc. Registry info

  31. Data Mining for Bioinformatics Community Registry Mining Factory Database Service BioDB 1 Compute Service Provider User Application . . . . . . “I want to create a personal database containing data on e.coli metabolism” Database Service Database Factory BioDB n Storage Service Provider

  32. Data Mining for Bioinformatics “Find me a data mining service, and somewhere to store data” Community Registry Mining Factory Database Service BioDB 1 Compute Service Provider User Application . . . . . . Database Service Database Factory BioDB n Storage Service Provider

  33. Data Mining for Bioinformatics Community Registry Mining Factory Database Service Handles for Mining and Database factories BioDB 1 Compute Service Provider User Application . . . . . . Database Service Database Factory BioDB n Storage Service Provider

  34. Data Mining for Bioinformatics Community Registry Mining Factory Database Service “Create a data mining service with initial lifetime 10” BioDB 1 Compute Service Provider User Application . . . . . . “Create a database with initial lifetime 1000” Database Service Database Factory BioDB n Storage Service Provider

  35. Data Mining for Bioinformatics Community Registry Mining Factory Database Service “Create a data mining service with initial lifetime 10” BioDB 1 Miner Compute Service Provider User Application . . . . . . “Create a database with initial lifetime 1000” Database Service Database Factory BioDB n Database Storage Service Provider

  36. Data Mining for Bioinformatics Community Registry Mining Factory Database Service Query BioDB 1 Miner Compute Service Provider User Application . . . . . . Query Database Service Database Factory BioDB n Database Storage Service Provider

  37. Data Mining for Bioinformatics Community Registry Mining Factory Database Service Query BioDB 1 Miner Keepalive Compute Service Provider User Application . . . . . . Query Database Service Database Factory Keepalive BioDB n Database Storage Service Provider

  38. Data Mining for Bioinformatics Community Registry Mining Factory Database Service BioDB 1 Miner Keepalive Compute Service Provider User Application . . . . . . Results Database Service Database Factory Keepalive Results BioDB n Database Storage Service Provider

  39. Data Mining for Bioinformatics Community Registry Mining Factory Database Service BioDB 1 Miner Compute Service Provider User Application . . . . . . Database Service Database Factory Keepalive BioDB n Database Storage Service Provider

  40. Data Mining for Bioinformatics Community Registry Mining Factory Database Service BioDB 1 Compute Service Provider User Application . . . . . . Database Service Database Factory Keepalive BioDB n Database Storage Service Provider

  41. OGSA Design & Implementation • OGSI (I=Infrastructure) WG in GGF defining core Grid service specification • (At least) three implementation efforts • Globus Toolkit => GT3 (alpha end 2002) • GT3 Core: Grid service specification • GT3 Base: Globus Toolkit behaviors • CIM resource model, GRAM-2 SLA negotiation, database services, … • Other GGF WGs address OGSA security, OGSA-compliant database services, etc.

  42. Overview • Origins: Resource sharing within scientific collaborations • Science drivers & science Grid projects • Globus Toolkit • Evolution: Resource virtualization • Commercial drivers • OGSA: Grid meets Web services • Futures • OGSA, GXA, and .NET

  43. 1) OGSA builds on infrastructure • Plumbing: WSDL, WS-Security, WS-Routing/Referral, reliable messaging, transactions, etc. • Hosting environments OGSI/OGSA Interfaces service description, service provisioning, … Standard OGSI container Web services various 2) to enable virtualization via • Service description • Service provisioning Hosting Environment Resource virtualization and QoS support 3) Standard container avoids implementing OGSI features in every service instance OGSA Implementation OGSI/OGSA Interfaces service description, service provisioning, … Standard OGSI container Web services various Hosting Environment Resource virtualization and QoS support

  44. OGSI, GXA, and .NET • Infrastructure • GXA Web services facilities provide OGSI plumbing • .NET provides nice hosting environment for Grid services • .NET also a good basis for standard OGSI container • OGSA can, in turn, provide resource description and provisioning for GXA/.NET OGSI/OGSA Interfaces service description, service provisioning, … Standard OGSI container Web services various GXA .NET Hosting Environment Resource virtualization and QoS support .NET

  45. Building an OGSI Container • Service data mgmt, query, subscription • Container should provide simple interface for interacting with an instance’s implementation to get and manage dynamic service data • Service instance = CLR object • Container should handle query processing • .NET support for XPath & Xquery allows for rich functionality • Container manages notification subscriptions, and drives asynchronous notification messages • Soft-state lifetime management • Soft-state registration

  46. Other Uses of .NET for OGSA • Resource description: WMI • WMI provides CIM-based infrastructure • Resource provisioning: various • Rich Windows support for QoS delivery • Real-time CPU scheduling • Network QoS (RSVP, diffserv, etc.) • Storage quotas? • Notification: MSMQ • Integrate with MSMQ for notification w/ QoS • Extensible factories: CLR

  47. Recap: Goals • Communicate the purpose, significance, state, adoption, & future of Grid technology • Persuade you that Grid technology represents an opportunity & challenge • Developers: Grids aren’t only (or particularly) about science and servers—themes of virtualization, service discovery, service management, and QoS delivery are universal • Researchers & educators: Rapid uptake in industry & science represents an exceptional opportunity for impact

  48. For More Information • The Globus Project™ • www.globus.org • Context & research articles • www.mcs.anl.gov/~foster • Open Grid Services Architecture • www.globus.org/ogsa • Global Grid Forum • www.gridforum.org • Edinburgh, July 22-24 • Chicago, Oct 15-17

More Related