assist e programmazione grid n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
ASSIST e Programmazione Grid PowerPoint Presentation
Download Presentation
ASSIST e Programmazione Grid

Loading in 2 Seconds...

play fullscreen
1 / 27

ASSIST e Programmazione Grid - PowerPoint PPT Presentation


  • 87 Views
  • Uploaded on

ASSIST e Programmazione Grid. Marco Vanneschi Dipartimento di Informatica Università di Pisa. Gruppo Assist, gennaio 2003. Segretaria del Gruppo: Simona Moi moi@di.unipi.it 050 - 2212718. Marco Aldinucci Sonia Campa Pierpaolo Ciullo Massimo Coppola Marco Danelutto Silvia Magini

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'ASSIST e Programmazione Grid' - oswald


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
assist e programmazione grid
ASSIST e Programmazione Grid

Marco Vanneschi

Dipartimento di Informatica

Università di Pisa

gruppo assist gennaio 2003
Gruppo Assist, gennaio 2003

Segretaria del Gruppo:

Simona Moi

moi@di.unipi.it

050 - 2212718

  • Marco Aldinucci
  • Sonia Campa
  • Pierpaolo Ciullo
  • Massimo Coppola
  • Marco Danelutto
  • Silvia Magini
  • Paolo Pesciullesi
  • Alessandro Petrocelli
  • Edoardo Pistoletti
  • Laura Potiti
  • Roberto Ravazzolo
  • Massimo Torquati
  • Marco Vanneschi
  • Corrado Zoccolo
obiettivo
Obiettivo
  • Caratteristiche base di ASSIST e possibili evoluzioni
  • Relazione con le esigenze della programmazione di griglie computazionali
  • Componenti e oggetti
  • Applicazioni ASSIST + CORBA
high performance grids high speed networks of of clusters of smp mpp
High-performance Grids: high-speed networks of … of clusters of … SMP/MPP ….
  • Uniform approach to
  • distribution + parallelism
  • Distributed Supercomputing,
  • On demand HPC,
  • Data Intensive Computing,
  • Collaborative Computing
  • Grid aware Computing
  • Parallel Programming as the basic methodology to understand and to realize development tools
    • to be characterized wrt the specific features of Grids
    • to be rendered consistent with the programmability, compositionality and reuse standards
uniform approach to distribution parallelism why
Uniform approach to distribution + parallelism: why ?
  • Not so obvious that distinct parallel components / objects can be efficiently allocated to distinct nodes
    • Optimization of computational and data resources
    • Evolution of networking and communication technology
  • Different versions of the same applications can be produced acting on parallelism degree, parallelism forms, multiplicity of components / objects
  • Not necessarily parallelism inside a component / object is fine grain; e.g. coarse grain farms
  • Adaptive applications
  • Data intensive applications
    • Interfacing data management to computations
slide6

Data intensive applications

Abstraction of Memory Hierarchy

Abstraction of Shared Objects

Component-structured application

Scheduling and configuration of complex, high-volume data flows through multiple levels of hierarchy.

assist
ASSIST

Structured Parallel Programming

+

High-performance Components

+

Objects

results of asi pqe project and agenzia 2000
Results of ASI-PQE project and Agenzia 2000
  • ASSIST version 1.0 for homogeneous and heterogeneous clusters (DI-UniPi, Synapsis)
    • A subset of ASSIST-CL
    • Support based on ACE + Distributed Shared Memory
    • Object-based design of compiler and support
    • Task-code and AssistLib: lower levels of programmability
  • Integration of scientific libraries in ASSIST support (CPS-CNR, Na)
  • AssistConf: a first tool for Grid configuration on top of Globus (ISTI-CNR, Pi)
  • Demonstration on computational chemistry applications (UniPg)and benchmarking (PoliMi)
assist coordination language assist cl
ASSIST Coordination Language (ASSIST-CL )
  • Programs as general graphs whose nodes are parallel modules and/or sequential modules
    • Compositionality through streams
    • Data-flow behaviour + state + nondeterminism
    • Programs reusable as nodes of other programs
  • Programs could also be expressed as “SkIE graphs” (pipeline,farm, loop skeletons), where the composed modules are ASSIST-modules
application structuring through parallel components and objects

Global variables

  • Shared memory
  • Files and I/O
  • Web
  • CORBA
  • ASSIST modules
  • . . .
  • External Objects 

Output stream

M1

s13

Input

streams

s34

M3

M4

M2

s23

s45

s54

M5

s25

Some abstractions are present in ASSIST 1.0, others will be realized in Grid projects

Component: Parallel (or sequential) module

Application structuring through Parallel Components and Objects
graphs and streams
Graphs and streams
  • ASSIST networks of components operate on streams
  • DAG as particular case (stream length = 1)
    • co-scheduling is expressed by program or implemented at the support level
    • in ASSIST DAGs could also be executed in pipeline
  • Importance of multiple streams
    • data intensive computations,
    • interactive computations,
    • calculation / communication overlapping,
formalisms for assist programs as a graph

s13

s34

M3

M4

M1

s23

s25

M2

M5

Formalisms for ASSIST programs as a graph
  • Graphs with general structure
  • Compositionality through streams (i.e. sequences of typed data)
  • Current, experimental syntax/semantics;
  • the FIRB GRID.IT project will consider
  • IDL-compliant
  • CCA-compliant
  • scripting languages
  • . . .
  • generic main (. . .)
  • stream int s13;
  • stream int [N][N] s23;
  • stream int [N][N] s34;
  • stream int s25;
    • M1 (ouput_stream s13);
    • M2 (output_stream s23, s25);
    • M3 (input_stream s13, s23; output_stream s34);
    • M4 (input_stream s34);
    • M5 (input_stream s25);
parallel module parmod parallel component in assist

External Objects

Input streams

Other Modules

Other

Modules

. . .

VP

VP

VP

. . .

VP

VP

VP

VP

VP

VP

Output streams

Several distribution and collection strategies, nondeterminism

Distributed internal state

Parallel Module(parmod): parallel component inASSIST

Set of Virtual Processors with names

parallel components objects

C1

C2

Ci

Cn

Parallel components + objects

S1

S2

. . .

Sj

. . .

Sm

External Objects

  • Not only a classical approach to software reuse
  • Often, external objects are simpler and/or more performant solutions to programming problems
    • e.g. data distribution, load balancing, memory capacity, …
    • dynamic data structures (shape and size not known until run-time)
  • In turn, parallel ASSIST programs may be used as objects

. . .

Components and streams

. . .

an example of complex application data mining c4 5 expressed as a parallel divide conquer
An example of complex application: data mining C4.5 expressed as a parallel Divide & Conquer

 training set TS, decision tree G 

For load balancing reasons,

during some phases the same parmod (Divide) works in a data-parallel manner,

in other phases in a farm-like manner,

in other phases …

Divide

Client

P1

Conquer

T

G1

G

G2

PN

G

Test

An example of adaptive strategy expressed at the programming level.

data intensive computations in assist

Object (possibly high-bandwidth)

External Object Interface (possibly parallel)

VP

VP

VP

Input Section

Output Section

VP

VP

VP

VP

VP

VP

ASSIST module (parallel component)

Data intensive computations in ASSIST

Abstraction of high-performance objects can be implemented by ASSIST parmod(s) with proper interface

(expressed in ASSIST or another formalism)

data intensive computations in assist1
Data intensive computations in ASSIST
  • High-performance abstaction of objects (e.g. complex memory hierarchies) can be implemented by parallelism (see previuos slide).
  • In some cases, it could be more efficient to interface external objects through the Input/Output sections of parallel components: an ASSISTextension to be studied.
  • The combination of techniques 1 and 2 might be very powerful.
input section virtual processors output section of parmod

External Objects

s13

I

Input Section

O

Output Section

s34

s23

  • For each output stream:
  • Output strategies of values computed by VPs
  • Possible post-elaborations and operations
  • Management of input streams according to the proper strategy (data-flow, nondeterministic)
  • Possible pre-elaborations (filter) and operations
  • Distribution of received values to VPs
Input Section, Virtual Processors, Output Section of parmod

VP

Virtual Processors

internal pipelining

input section data distributions
Input Section: data distributions

+ nondeterminism and “on the fly” processing of stream data

  • On demand
  • Broadcast
  • Scatter
  • Multicast

Input/Output sections could be optimized for external objects interfacing

  • data management, memory hierarchies
  • minimizing bottlenecks (pre/post-processing)
  • in addition to VP parallel access to objects
  • security and fault tolerance
optimizations scheduling mapping firb grid it project
Optimizations, scheduling, mapping, …(FIRB GRID.IT project)
  • New research issues [e.g.: GrADS] are to be dealt with:
    • challanges in static + dynamic compiler design
    • importance of a “constraint-based” programming model
  • New impacts on middleware
    • new light-weight middleware
    • exploiting Knowledge Discovery in Grid databases
  • More that one level of programmability?
  • Expressive power for dynamic behaviours.
assist e corba
ASSISTeCORBA
  • Un programma ASSIST può importare ed esportare servizi tramite lo standard CORBA.
  • ASSIST vede CORBA come un oggetto esterno
  • L’importazione/esportazione avviene all’interno delle procedure (proc) di un qualsiasi modulo sequenziale o parallelo.
  • L’utilizzo di CORBA all’interno di un programma ASSIST non richiede particolari accorgimenti sintattici.
cliente in assist

Server

CORBA

Client ASSIST

Cliente in ASSIST
  • Il Server CORBA esporta un servizio che si può utilizzare in un programma ASSIST.
  • Il Cliente è un modulo ASSIST (sequenziale o parallelo) che:
    • inizializza un ORB
    • si procura un object Reference
    • invoca il metodo del server.
cliente parallelo

GUI

Server CORBA

Cliente parallelo

Programma ASSIST

client

loop

N-body

  • Il programma ASSIST ottiene i dati di input dal server CORBA - GUI
  • Il programma ASSIST esegue N-body e visualizza i dati tramite il server CORBA.
  • L’interazione avviene quindi contemporaneamente con il calcolo
esportazione di servizi
Esportazione di servizi
  • Ogni applicazione Fscritta in ASSIST, con un INPUT (A) ed un OUTPUT (B),
    • puo’ essere esportata come servizio attraverso CORBA,
    • aggiungendo all’applicazione un modulo che svolga le funzioni del Server.
  • Il Server esporta il metodo do_F con l’INPUT e l’OUTPUT dell’applicazione F.

Programma ASSIST

A

server

F

B

Programma ASSIST disponibile come servizio CORBA

esempio di esportazione di servizi
Esempio di esportazione di servizi
  • SPT: parmod ASSIST
  • Server: modulo ASSIST
  • che si interfaccia con il
  • mondo esterno mediante
  • CORBA.
    • Il server puo’ essere
  • generato in modo
  • automatico a partire
  • dal codice ASSIST.
  • Client: programma scritto
  • in un qualsiasi linguaggio supportato da CORBA;
    • invoca il metodo do_spt esportato dal server.

Client con interfaccia CORBA

B = do_SPT(A)

Programma ASSIST

A

server

spt

B

parmod

one

parmod

array