1 / 92

Grid Architecture

Grid Architecture. Prof. Ruay-Shiung Chang March 2004. Course Contents. Grid Architecture Open Grid Services Architecture Resource and Service Management Building Reliable Clients and Services Instrumentation and Monitoring. Grid Architecture. Grid Architecture.

devika
Download Presentation

Grid Architecture

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Grid Architecture Prof. Ruay-Shiung Chang March 2004

  2. Course Contents • Grid Architecture • Open Grid Services Architecture • Resource and Service Management • Building Reliable Clients and Services • Instrumentation and Monitoring

  3. Grid Architecture

  4. Grid Architecture

  5. Why Discuss Architecture? • Descriptive • Provide a common vocabulary for use when describing Grid systems • Guidance • Identify key areas in which services are required • Prescriptive • Define standard “Intergrid” protocols and APIs to facilitate creation of interoperable Grid systems and portable applications

  6. Architecture is to make us know and remember who we are. - Sir Geoffrey Jellicoe

  7. Water Garden in England

  8. One View of Requirements • Adaptation • Intrusion detection • Resource management • Accounting & payment • Fault management • System evolution • etc. • etc. • … • Identity & authentication • Authorization & policy • Resource discovery • Resource characterization • Resource allocation • (Co-)reservation, workflow • Distributed algorithms • Remote data access • High-speed data transfer • Performance guarantees • Monitoring

  9. Another View: “Three Obstaclesto Making Grid Computing Routine” • New approaches to problem solving • Data Grids, distributed computing, peer-to-peer, collaboration grids, … • Structuring and writing programs • Abstractions, tools • Enabling resource sharing across distinct institutions • Resource discovery, access, reservation, allocation; authentication, authorization, policy; communication; fault detection and notification; …

  10. The greatest obtacle to progress is prejudice. - Bavee

  11. The Systems Problem:Resource Sharing Mechanisms That … • Address security and policy concerns of resource owners and users • Are flexible enough to deal with many resource types and sharing modalities • Scale to large number of resources, many participants, many program components • Operate efficiently when dealing with large amounts of data & computation

  12. Aspects of the Systems Problem • Need for interoperability when different groups want to share resources • Diverse components, policies, mechanisms • E.g., standard notions of identity, means of communication, resource descriptions • Need for shared infrastructure services to avoid repeated development, installation • E.g., one port/service/protocol for remote access to computing, not one per tool/app • E.g., Certificate Authorities: expensive to run • A common need for protocols & services

  13. Hence, a Protocol-Oriented Viewof Grid Architecture, that Emphasizes … • Development of Grid protocols & services • Protocol-mediated access to remote resources • New services: e.g., resource brokering • “On the Grid” = speak Intergrid protocols • Mostly (extensions to) existing protocols • Development of Grid APIs & SDKs • Interfaces to Grid protocols & services • Facilitate application development by supplying higher-level abstractions • The (hugely successful) model is the Internet

  14. Application Application Internet Protocol Architecture “Coordinating multiple resources”: ubiquitous infrastructure services, app-specific distributed services Collective “Sharing single resources”: negotiating access, controlling use Resource “Talking to things”: communication (Internet protocols) & security Connectivity Transport Internet “Controlling things locally”: Access to, & control of, resources Fabric Link Layered Grid Architecture(By Analogy to Internet Architecture)

  15. Protocols, Services,and APIs Occur at Each Level Applications Languages/Frameworks Collective Service APIs and SDKs Collective Service Protocols Collective Services Resource APIs and SDKs Resource Service Protocols Resource Services Connectivity APIs Connectivity Protocols Local Access APIs and Protocols Fabric Layer

  16. Important Points • Built on Internet protocols & services • Communication, routing, name resolution, etc. • “Layering” here is conceptual, does not imply constraints on who can call what • Protocols/services/APIs/SDKs will, ideally, be largely self-contained • Some things are fundamental: e.g., communication and security • But, advantageous for higher-level functions to use common lower-level functions

  17. The Hourglass Model • Focus on architecture issues • Propose set of core services as basic infrastructure • Use to construct high-level, domain-specific solutions • Design principles • Keep participation cost low • Enable local control • Support for adaptation • “IP hourglass” model A p p l i c a t i o n s Diverse global services Core services Local OS

  18. Hourglass

  19. Where Are We With Architecture? • No “official” standards exist • But: • Globus Toolkit™ has emerged as the de facto standard for several important Connectivity, Resource, and Collective protocols • GGF has an architecture working group • Technical specifications are being developed for architecture elements: e.g., security, data, resource management, information • Internet drafts submitted in security area

  20. Fabric LayerProtocols & Services • Just what you would expect: the diverse mix of resources that may be shared • Individual computers, Condor pools, file systems, archives, metadata catalogs, networks, sensors, etc., etc. • Few constraints on low-level technology: connectivity and resource level protocols form the “neck in the hourglass” • Defined by interfaces not physical characteristics

  21. Fabrics in general

  22. Connectivity LayerProtocols & Services • Communication • Internet protocols: IP, DNS, routing, etc. • Security: Grid Security Infrastructure (GSI) • Uniform authentication, authorization, and message protection mechanisms in multi-institutional setting • Single sign-on, delegation, identity mapping • Public key technology, SSL, X.509, GSS-API • Supporting infrastructure: Certificate Authorities, certificate & key management, …

  23. Not too many connections!

  24. Resource LayerProtocols & Services • Grid Resource Allocation Management (GRAM) • Remote allocation, reservation, monitoring, control of compute resources • GridFTP protocol (FTP extensions) • High-performance data access & transport • Grid Resource Information Service (GRIS) • Access to structure & state information • Others emerging: Catalog access, code repository access, accounting, etc. • All built on connectivity layer: GSI & IP

  25. Collective LayerProtocols & Services • Index servers aka metadirectory services • Custom views on dynamic resource collections assembled by a community • Resource brokers (e.g., Condor Matchmaker) • Resource discovery and allocation • Replica catalogs • Replication services • Co-reservation and co-allocation services • Workflow management services • etc. Condor: www.cs.wisc.edu/condor

  26. Collectives: The Borgs Resistance is futile. You will be assimilated.

  27. API SDK C-point Protocol Checkpoint Repository API SDK Access Protocol Compute Resource Example: High-Throughput Computing System App High Throughput Computing System Collective (App) Dynamic checkpoint, job management, failover, staging Collective (Generic) Brokering, certificate authorities Access to data, access to computers, access to network performance data Resource Communication, service discovery (DNS), authentication, authorization, delegation Connect Storage systems, schedulers Fabric

  28. Example:Data Grid Architecture App Discipline-Specific Data Grid Application Collective (App) Coherency control, replica selection, task management, virtual data catalog, virtual data code catalog, … Collective (Generic) Replica catalog, replica management, co-allocation, certificate authorities, metadata catalogs, Access to data, access to computers, access to network performance data, … Resource Communication, service discovery (DNS), authentication, authorization, delegation Connect Storage systems, clusters, networks, network caches, … Fabric

  29. CERN’s Data Grid Application

  30. The Compact Muon Solenoid

  31. Open Grid Service Architecture Minds are like parachutes, they only function when they are open. – Thomas Robert Dewar

  32. Service-Oriented Architecture • Service: an entity that provides some capability to its clients by exchanging messages • Service: defined by identifying sequences of specific message exchanges that cause the service to perform some operations

  33. Service-oriented Architecture • Great flexibility in implemen-tation and location: because all operations are defined in terms of message exchanges SOAP: Simple Object Access Protocol

  34. Service-oriented Architecture • Definition: a service-oriented architecture is one in which all entities are services, and thus any operation visible to the architecture is the result of message exchange services Message exchanges Underlying infrastructure

  35. Service-oriented architecture • Examples: • A storage service • A data transfer service • A troubleshooting service • Two important themes low-level service high-level service

  36. But, first four personality types

  37. Service-oriented architecture • Common behaviors can reoccur in different contexts • A goal of OGSA design is to allow these behaviors to be expressed in standard ways regardless of contexts, so as to simplify application design and encourage code reuse

  38. Service-oriented architecture • A higher-level service behavior (data transfer) can be implemented via the composition of simpler behaviors (storage service) • Ease of composition is a second major design goal for OGSA

  39. Service-oriented architecture • Similarity in protocols

  40. Service-oriented architecture • By encapsulating service operations behind a common message-oriented service interface, service-oriented architecture encourages service virtualization, isolating users from details of service implantation and location

  41. Service-oriented architecture Virtualization

  42. Service-oriented architecture • Virtualization • Everything is becoming virtual: virtual stores, virtual workspace, virtual organization, virtual networks, … • More than 60 million US workers work remotely

  43. Service-oriented architecture • Interaction with a given service is facilitated by using a standard interface definition language (IDL), such as WSDL, to describe the service’s interfaces.

  44. Service-oriented architecture • Web service description language

  45. Service-oriented architecture • IDL: a cornerstone of interoperability and transparency All I really need to know I learned in kindergarten. from interfaces

  46. Service-oriented architecture • An IDL defines the operations supported by a service, by specifying the messages that a service consumes and produces • An interface specification describes the messages the service expects but does not define what the service does in response to those messages (i.e., its behavior)

  47. Service-oriented architecture

More Related