Tools and utilities for parallel and serial codes in enea grid environment
Download
1 / 17

Tools and Utilities for parallel and serial codes in ENEA-GRID environment - PowerPoint PPT Presentation


  • 106 Views
  • Uploaded on
  • Presentation posted in: General

C.R. ENEA-Portici. 11/12/2007. Tools and Utilities for parallel and serial codes in ENEA-GRID environment. CRESCO Project: Salvatore Raia SubProject I.2. OUTLINE:. GRID, cluster and parallel Computing (Intro) ENEA-GRID. Architecture and functionality

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha

Download Presentation

Tools and Utilities for parallel and serial codes in ENEA-GRID environment

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


C.R. ENEA-Portici. 11/12/2007

Tools and Utilities for parallel and serial codes in ENEA-GRID environment

CRESCO Project: Salvatore Raia

SubProject I.2


OUTLINE:

  • GRID, cluster and parallel Computing (Intro)

  • ENEA-GRID. Architecture and functionality

  • My Activity for CRESCO project and results on ENEA-GRID

  • Conclusion and objectives

C.R. ENEA-Portici. 11/12/2007


Collection of resources (HW, SW) connected via public or private network - Each CPU runs a separated istance of operating system-Administration: local

What is a cluster ?

Supercomputer= computer with many processors connected via high-speed computer bus and that share the memory (SMP) . It runs one Operating system

cluster 1

supercomputer

C.R. ENEA-Portici. 11/12/2007


cluster 1

cluster 2

cluster 3

cluster N

GRID 1

Collection of interconnected clusters geographically distributed - administration: sometimes clusters belong to different department or company

How to get a Grid ?

GRID = nodes made of clusters and

each node may have Shared or

Distributed memory architectures

(Hybrid ) that share processes .

ENEA-GRID has the same structure

With 6 clusters: Bologna, Casaccia,

Frascati, Portici, Trisaia, Brindisi

C.R. ENEA-Portici. 11/12/2007


ENEA-GRID structure (HW)

C.R. ENEA-Portici. 11/12/2007


Pro:

Shared resources

Low costs (clock ?)

Open systems

Scalability

Con:

Several platforms

Load balancing

User Access

Frequency scaling (domain ?)

Power consumption P=C×V×V×F

How is it managed on

ENEA-GRID ?

GRID features

C.R. ENEA-Portici. 11/12/2007


ENEA-GRID structure (SW)

Resources management

ICA client

File System

Operating Systems

C.R. ENEA-Portici. 11/12/2007


Switch host

Run Appl.

Jobs status

User Interface

USER ACCESS

  • ICA client

  • ssh o telnet

  • web

C.R. ENEA-Portici. 11/12/2007


Problem with:

Multi platforms

Load balancing

User Access

How to cope with ?

LSF utilities

Software dev.

User interfaces

My activity on ENEA-GRID (CRESCO pr.)

Serial and Parallel (MPI) codes

C.R. ENEA-Portici. 11/12/2007


Serial codes

Compilers

GNU

PGI

IBM

Parallel codes (MPI)

MPI Implementations

MPICH

LAM-MPI

POE

Problems with execution too

Tools for Serial and Parallel (MPI) codes

Multi Platform

…So we need a lots of binaries for each platform.

Launcher: after compiling our source code in each platform, we

have “binary1”…”binaryN” for host1,…hostN.

It is a shell script (placed on AFS) that selects the righteous

“binary” for the selected host

…tools

C.R. ENEA-Portici. 11/12/2007


Some MPI problems

C.R. ENEA-Portici. 11/12/2007


Results: tools serial and parallel (MPI) codes

  • SERIAL

    • Program for Fortran 77/90,C and C++ serial compiling

      (look Java Interface)

    • Launcher for “NS2” application (use external libraries)

  • PARALLEL (MPI)

    • Launcher for running a test program (check command)

    • Launcher for HPL test on AIX and Linux

user1 installation

user2 installation

C.R. ENEA-Portici. 11/12/2007


Serial codes

Resources definition

“NS2” application

Serial LSF utilities

Job array (Multicase)

“lsgrun”

Parallel codes (MPI)

Parallel LSF utilities

“mpijob” (MPICH)

“poejob” (POE)

No correlation

Correlation

Analizing LSF utilities Serial and Parallel codes

LSF Resources

C.R. ENEA-Portici. 11/12/2007


Serial codes

Parallel codes (MPI)

Results: Integration with other application

(My)Java Interface

C.R. ENEA-Portici. 11/12/2007


Conclusion and objectives

  • Launcher + LSF utilities + User interface

    allow to create a omogeneous environment

  • Objectives:

    • Optimization of programs to launch serial and parallel codes, including checking resources to run the application (e.g. library, other programs, etc)

    • Exploitation of LSF utilities in order to make easy running MPI programs (mpijob, poejob, etc) and load balancing

    • Improve error handling for user interfaces … …

C.R. ENEA-Portici. 11/12/2007


Andrew File System

C.R. ENEA-Portici. 11/12/2007


LSF-Load Sharing Facilities

C.R. ENEA-Portici. 11/12/2007


ad
  • Login