1 / 20

PAGIS: An Architecture for Programming on the Grid

PAGIS: An Architecture for Programming on the Grid. Andrew Wendelborn Distributed & High Performance Computing Group Department of Computer Science, University of Adelaide Adelaide SA 5005, Australia Email: andrew@cs.adelaide.edu.au December 2002. Background. PAGIS DHPC propject

krista
Download Presentation

PAGIS: An Architecture for Programming on the Grid

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PAGIS: An Architecture for Programming on the Grid Andrew Wendelborn Distributed & High Performance Computing Group Department of Computer Science, University of Adelaide Adelaide SA 5005, Australia Email: andrew@cs.adelaide.edu.au December 2002

  2. Background • PAGIS • DHPC propject • Processing of satellite imagery • Started as a compositional process-oriented model for GIS applications • Now an architecture for building grid applications • Outline • Base-level API and GUI • Reflective (metalevel) programming for grid applications • Framework for defining grid behaviours • Adaptive software: safe reconfiguration • An adaptive resource broker

  3. Principles • High-level computational model • Programmers shouldn’t need to be grid experts • Reason about behaviour at an abstract level • Simple, intuitive interface • Following compositional component-based paradigm • Access to system • Programmer abstracts over low-level details … BUT • Wants to tune aspects of system critical to grid application • We provide this via a separate interface

  4. Process Networks • General, clearly defined semantics • Separates meaning from implementation • Different strategies produce equivalent results • Compositional • Reconfigurable • Highly applicable to grid programming Process2 Process4 Process1 Process3

  5. Client PAGIS overview GUI PN structure Evaluator

  6. Client Scheduler Evaluator

  7. Client Grid enabled Globus toolkit Distributed JVMs Scheduler GIIS GRAM - GRIS GRAM - GRIS Evaluator

  8. Tuning and Behaviour • We have seen how to build base-level application • How to fine-tune its implementation and behaviour • Types of tuning • Replacing code for PN process • Allocation and re-allocation • Monitoring, and other non-application (meta) concerns • “Actors” observe and influence behaviour • Human observation and intervention • Other software systems • Change how the computation occurs, not the result of the computation

  9. A Reflective Approach • Separate concerns into two programs • Functional concerns: baselevel program • Non-functional concerns: metalevel program (meta-behaviour) • Reflects separation in process network model • Reify certain aspects of baselevel • Metalevel programmer reflects upon selected aspects to control or customize aspects critical to application • Software engineering for the grid

  10. MI Method Reification • Constructs an explicit representation of a method invocation at the metalevel • Provides hooks to control and customize method invocation process • We have designed a customization framework, Enigma MetaA MetaB ObjectA ObjectB

  11. Grid Behaviours • Enigma gives us a model for describing and implementing interesting grid behaviours • Locality • Migrating the computation and its parts • Allocation and Adaptation • Computation meta-attributes to drive allocator • Periodically reassess allocation • Performance Monitoring • Intercept invocations to monitor progress

  12. Process Network API Enigma Metalevel Interface PN System Implementation PAGIS as a Grid Programming Model • API provides baselevel interface to system • PN provides semantics and safety of adaptation • Enigma: hooks to observe / influence behaviour • Software engineering of adaptive applications

  13. The System Client • Java implementation • Globus toolkit • Java CoG • Coglets for peer-to-peer Java / Globus Manager Allocation Globus Plugin Client Plugin Distribution Tools Channel Service Process Service COG Broker RSL LDAPquery Gatekeeper Worker MDS Job Manager GRAM GRAMdata Worker JVM fork

  14. Current Implementation • Building a PN • Java; CoG; Globus toolkit • Operations for simple image processing • Inspection of meta-level attributes • Changing meta-level attributes • Influences allocation • May lead to migration • Underway: integration with DARC

  15. PAGIS and DARC Host A Host B DR TCP DR Node • DARC provides data abstractions • PAGIS provides process abstractions Node DR DR Node DR DR Host C

  16. DARC provides data sources Powerful abstraction over Globus toolkit Client Scheduler GIIS GRAM - GRIS DARC Evaluator

  17. Adaptive Resource Broker Project • Testbed for different RB strategies • Emulator (NISTNet) and simulators • NWS (Network Weather Service) • Performance database • Forecasts • Detection of anomalous conditions • Generate threshold events • RB adapts its behavior

  18. Possible emulation framework

  19. Grid Simulator • Simple discrete event simulation package • Builds on GridSim and SimGrid • Java, multi-threaded • Can specify a full network topology • Heterogeneous resources of any kind modelled easily • Arbitrary information can be published about resources • Allows study of RB techniques on an arbitrary grid • monitored by NWS • threshold events feedback to RB

  20. Next Steps • Java peer-to-peer environment • Coglets for peer-to-peer Java / Globus • DARC integration • Adaptive middleware for mobile computing • Experimental testbed for Enigma-based behaviours • Adaptive RB framework • XML intermediate form for PN structure • … applications …

More Related