components for high performance grid programming in the grid it project
Download
Skip this Video
Download Presentation
Components for high performance grid programming in the GRID.IT project

Loading in 2 Seconds...

play fullscreen
1 / 26

Components for high performance grid programming in the GRID.IT project - PowerPoint PPT Presentation


  • 90 Views
  • Uploaded on

Components for high performance grid programming in the GRID.IT project. M. Aldinucci M. Coppola D. Laforenza D. Puppin ISTI CNR Pisa. S. Campa M. Danelutto L. Scarponi M. Vanneschi C. Zoccolo Dept. Computer Science University of Pisa. The goal.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Components for high performance grid programming in the GRID.IT project' - kamran


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
components for high performance grid programming in the grid it project

Components for high performance grid programming in the GRID.IT project

M. Aldinucci

M. Coppola

D. Laforenza

D. Puppin

ISTI

CNR Pisa

S. Campa

M. Danelutto

L. Scarponi

M. Vanneschi

C. Zoccolo

Dept. Computer Science

University of Pisa

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

the goal
The goal
  • Programming environment supporting the development of
    • Efficient
    • Scalable
    • Fault tolerant
    • Adaptive
  • grid aware applications

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

starting from assist
Starting from: ASSIST
  • Structured parallel programming environment
  • Sequential (C, C++, F77) or parallel modules
  • Data flow streams (nondeterministic control)
  • Generic graph compositions
  • Parallel module:parmod
    • User defined: input/output section, virtual processors
    • Specialized to implement most classical and non classical parallelism exploitation patterns

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

the assist track
The ASSIST track …
  • ASSIST 1.0
    • Targets homogeneous WS clusters (POSIX TCP/IP)
  • ASSIST 1.2
    • Heterogeneous clusters (Linux Pentium / MacOSX PPC) and GRID (Globus 2.4)
  • ASSIST 2.0 (under development)
    • GRID, adaptivity, components
    • … and most of this talk!

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

the approach layered architecture
Applications

Programming environment

Abstract machine

GRID middleware

The approach: layered architecture
  • Applications
    • Complex, multidisciplinary, parallel
    • High level programming tools
  • High performance programming environment
    • Structured, compositional, adaptive
    • Performance contracts
  • Grid abstract machine
    • Functionalities from resource, collective and connectivity
    • Middleware independent API

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

programming model
Programming model
  • Component based
    • Inherits from existing ones (CCA, CCM, Proactive, …)
  • High performance
    • Builds on efficient mechanisms
  • Parallel
    • High level programming mechanisms and tools
  • Interoperability
    • Standard component frameworks
  • Code reuse
    • Sequential code in components

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

computational model parallelism
Computational model: parallelism
  • Structured parallelism exploitation
  • Programmers  quality of parallelism + contracts
  • Environment (compiler + run time)  mechanisms, quantity, heuristics

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

computational model interaction
Computational model: interaction
  • Compositionality of components
  • Classic mechanisms:
    • RPC/RMI
    • Events
  • New mechanisms:
    • Data flow streams
      • Non deterministic control
      • Component reacts to incoming data

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

computational model adaptivity
Computational model: adaptivity
  • Performance contracts
    • Specified by the programmer
    • Performance requirements (bandwidth, parallelism degree, …)
  • Restructuring strategies
    • Specified by user
    • Program rewriting (different parallel patterns, parallelism degree, …)

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

comp model adaptivity 2
Comp. Model: adaptivity (2)
  • Allocation constrains
    • Mapping computations to nodes
    • Data access constrains
  • Adaptivity in the environment:
    • Monitoring of activities
    • Triggering of corrective actions
    • Notifying the user

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

component model
Non functional interface

(contracts, adaptivity)

Non functional interface

(contracts, adaptivity)

Existing code reuse

Parallel structure

Existing code reuse

Parallel structure

Seq. comp.

Par. comp.

legacy comp.

New mechanisms (data flow streams)

New mechanisms (data flow streams)

Standard interface(uses/provides, events)

Standard interface(uses/provides, events)

Component model

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

application manager
ASSIST components

GAM

Application manager (AM)

Abstraction of basic services

Standard middleware

Application manager
  • High performance components
  • ASSIST programs

Grid Abstract Machine (GAM)

Standard middleware

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

application manager 2
C0

C2

C4

C0

C1

C2

C4

C5

C1

C3

C5

C3

AM

MAM1

MAM2

MAM3

MAM4

MAM5

AM

MAM1

MAM2

MAM3

MAM4

MAM5

CAM01

CAM02

CAM0

CAM0

Application manager (2)

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

module application manager
Module Application Manager
  • Configuration control of single ASSIST parmod
  • Exploits Performance model repository
    • Classical models
    • History
    • Compiler or programmer responsibility
  • Adapts the parmod behaviour to satisfy module performance contract

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

performance model repository
Performance Model Repository
  • Classical models
    • Parallelism degree = Compute time / Communication time
  • Dynamic models
    • Increase parallelism degree step-by-step until results start coming back
  • User provided
    • Proper formalism

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

component application manager
Component Application Manager
  • Global control strategies
  • Inputs from MAMs
  • Possibily overcomes MAMs proposals
    • E.g. 2 stage pipeline
      • First stage achieves performance contract ++ (very good resource recruited)
      • CAM requires new contract to the second stage

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

reconfiguration strategies
Reconfiguration strategies
  • C1 virtualizes grid data resource in two streams
  • C2 is sequential by default strategy tells  farm
  • C3 data parallelstrategy tells parallelism degree
  • C4 is sequential by default strategy tells if(Condx) data parallel else farm
  • C5 non det mapping of streams to data set
  • C2 bottleneck!
    • C2 new farm with proper parallelism degree
    • C4 C5 restructured as a consequence

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

experiments 1
Experiments (1)

ASSIST

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

experiments 2
Experiments (2)

ASSIST

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

experiments 3
Experiments (3)

ASSIST

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

experiments 4
Experiments (4)

ASSIST

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

experiments 5
Experiments (5)

Lithium

muskel

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

experiments 6
Experiments (6)

Lithium

muskel

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

experiments 7
Experiments (7)

Lithium

muskel

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

conclusions
Conclusions
  • On going work
  • In the framework of GRID.it project
    • Involving major research institutions (grid, high performance, parallelism) in Italy
  • Main contribution
    • Non functional interfaces + application managers
  • Currently being experimented/implemented on/in ASSIST

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

slide26
Any questions ?

di.unipi.it

[email protected]

isti.cnr.it

Workshop on Component Models and Systems for Grid Applications - St.Malo 26 june 2004

ad