1 / 26

Components for high performance grid programming in the GRID.IT project

Components for high performance grid programming in the GRID.IT project. M. Aldinucci M. Coppola D. Laforenza D. Puppin ISTI CNR Pisa. S. Campa M. Danelutto L. Scarponi M. Vanneschi C. Zoccolo Dept. Computer Science University of Pisa. The goal.

kamran
Download Presentation

Components for high performance grid programming in the GRID.IT project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Components for high performance grid programming in the GRID.IT project M. Aldinucci M. Coppola D. Laforenza D. Puppin ISTI CNR Pisa S. Campa M. Danelutto L. Scarponi M. Vanneschi C. Zoccolo Dept. Computer Science University of Pisa Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  2. The goal • Programming environment supporting the development of • Efficient • Scalable • Fault tolerant • Adaptive • grid aware applications Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  3. Starting from: ASSIST • Structured parallel programming environment • Sequential (C, C++, F77) or parallel modules • Data flow streams (nondeterministic control) • Generic graph compositions • Parallel module:parmod • User defined: input/output section, virtual processors • Specialized to implement most classical and non classical parallelism exploitation patterns Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  4. The ASSIST track … • ASSIST 1.0 • Targets homogeneous WS clusters (POSIX TCP/IP) • ASSIST 1.2 • Heterogeneous clusters (Linux Pentium / MacOSX PPC) and GRID (Globus 2.4) • ASSIST 2.0 (under development) • GRID, adaptivity, components • … and most of this talk! Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  5. Applications Programming environment Abstract machine GRID middleware The approach: layered architecture • Applications • Complex, multidisciplinary, parallel • High level programming tools • High performance programming environment • Structured, compositional, adaptive • Performance contracts • Grid abstract machine • Functionalities from resource, collective and connectivity • Middleware independent API Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  6. Programming model • Component based • Inherits from existing ones (CCA, CCM, Proactive, …) • High performance • Builds on efficient mechanisms • Parallel • High level programming mechanisms and tools • Interoperability • Standard component frameworks • Code reuse • Sequential code in components Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  7. Computational model: parallelism • Structured parallelism exploitation • Programmers  quality of parallelism + contracts • Environment (compiler + run time)  mechanisms, quantity, heuristics Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  8. Computational model: interaction • Compositionality of components • Classic mechanisms: • RPC/RMI • Events • New mechanisms: • Data flow streams • Non deterministic control • Component reacts to incoming data Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  9. Computational model: adaptivity • Performance contracts • Specified by the programmer • Performance requirements (bandwidth, parallelism degree, …) • Restructuring strategies • Specified by user • Program rewriting (different parallel patterns, parallelism degree, …) Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  10. Comp. Model: adaptivity (2) • Allocation constrains • Mapping computations to nodes • Data access constrains • Adaptivity in the environment: • Monitoring of activities • Triggering of corrective actions • Notifying the user Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  11. Non functional interface (contracts, adaptivity) Non functional interface (contracts, adaptivity) Existing code reuse Parallel structure Existing code reuse Parallel structure Seq. comp. Par. comp. legacy comp. New mechanisms (data flow streams) New mechanisms (data flow streams) Standard interface(uses/provides, events) Standard interface(uses/provides, events) Component model Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  12. ASSIST components GAM Application manager (AM) Abstraction of basic services Standard middleware Application manager • High performance components • ASSIST programs Grid Abstract Machine (GAM) Standard middleware Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  13. C0 C2 C4 C0 C1 C2 C4 C5 C1 C3 C5 C3 AM MAM1 MAM2 MAM3 MAM4 MAM5 AM MAM1 MAM2 MAM3 MAM4 MAM5 CAM01 CAM02 CAM0 CAM0 Application manager (2) Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  14. Module Application Manager • Configuration control of single ASSIST parmod • Exploits Performance model repository • Classical models • History • Compiler or programmer responsibility • Adapts the parmod behaviour to satisfy module performance contract Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  15. Performance Model Repository • Classical models • Parallelism degree = Compute time / Communication time • Dynamic models • Increase parallelism degree step-by-step until results start coming back • User provided • Proper formalism Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  16. Component Application Manager • Global control strategies • Inputs from MAMs • Possibily overcomes MAMs proposals • E.g. 2 stage pipeline • First stage achieves performance contract ++ (very good resource recruited) • CAM requires new contract to the second stage Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  17. Reconfiguration strategies • C1 virtualizes grid data resource in two streams • C2 is sequential by default strategy tells  farm • C3 data parallelstrategy tells parallelism degree • C4 is sequential by default strategy tells if(Condx) data parallel else farm • C5 non det mapping of streams to data set • C2 bottleneck! • C2 new farm with proper parallelism degree • C4 C5 restructured as a consequence Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  18. Experiments (1) ASSIST Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  19. Experiments (2) ASSIST Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  20. Experiments (3) ASSIST Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  21. Experiments (4) ASSIST Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  22. Experiments (5) Lithium muskel Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  23. Experiments (6) Lithium muskel Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  24. Experiments (7) Lithium muskel Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  25. Conclusions • On going work • In the framework of GRID.it project • Involving major research institutions (grid, high performance, parallelism) in Italy • Main contribution • Non functional interfaces + application managers • Currently being experimented/implemented on/in ASSIST Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

  26. Any questions ? di.unipi.it First.Family@ isti.cnr.it Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

More Related