libraries for clustered computing and a few other things
Download
Skip this Video
Download Presentation
Libraries for Clustered Computing and a few other things ...

Loading in 2 Seconds...

play fullscreen
1 / 44

Libraries for Clustered Computing - PowerPoint PPT Presentation


  • 182 Views
  • Uploaded on

Libraries for Clustered Computing and a few other things . Anne E. Trefethen NAG Ltd, Oxford. What is NAG?. Mishi Derakhshan Nadir Bouhamou Vince Fernando Arnold Krommer. Stefano Salvini Lawrence Mulholland Sven Hammarling Jeremy DuCroz Ken McDonald. Acknowledgements.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Libraries for Clustered Computing' - RexAlvis


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
what is nag
What is NAG?

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

acknowledgements
Mishi Derakhshan

Nadir Bouhamou

Vince Fernando

Arnold Krommer

Stefano Salvini

Lawrence Mulholland

Sven Hammarling

Jeremy DuCroz

Ken McDonald

Acknowledgements

The ScaLAPACK team

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

nag involvement in high performance computing
NAG Involvement in High Performance Computing

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

a couple of general points
A Couple of General Points
  • Important to NAG that libraries are portable across variety of platforms => Standards
  • The top 1% of machines are interesting but NAG needs to focus on a broader user base.

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

nag s approach to hpc
NAG’s approach to HPC

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

requirements from applications
Requirementsfrom Applications
  • Algorithms for variety of memory hierarchies
  • Implementations for a variety of platforms
    • Accessible from a range of languages
    • Conforming with different parallel paradigms
  • Clean application interface
  • Tested and reliable

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

nag parallel library
NAG Parallel Library
  • Scope
    • Library of parallel numerical subroutines
    • Assortment of support routines
  • Target: any parallel system with MPI/PVM
    • shared/distributed memory computers
    • networks/clusters of workstations
  • Goals: effectiveuse of multiple processors
  • effective use of distributed memory

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

nag parallel library design
NAG Parallel Library - Design
  • Ease of use: routine interfaces
    • similar to sequential NAG Fortran routines
    • adhere to Single Program Multiple Data (SPMD) paradigm
    • provide distribution/in-place generation routines
  • Portability
    • Communication: BLACS + MPI/PVM
    • Computation: Fortran 77 + ANSI C

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

slide10

ScaLAPACK Structure

ScaLAPACK

PBLAS

LAPACK

BLACS

BLAS

PVM/MPI/...

Global

Local

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

nag parallel library design11
NAG Parallel Library - Design
  • Performance
    • computational routines assume data is in-place
    • link to machine-optimized BLAS, PBLAS, etc.
    • relaxed error checking
  • Reliability
    • NAG or LAPACK type error checking
    • global argument checking & error parameter
    • adhere to stringent NAG quality assurance standards

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

slide12

Distributed Memory Parallel Library

Matrix Distribution

Quadrature

Eigenproblems

Iterative Solvers

I/O utilities

Optimization

Random numbers

Direct Solvers (ScaLAPACK)

Least-Squares (ScaLAPACK)

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

p arallel i ndustrial n um e rical a pplications and p ortable l ibraries

P I N E A P L

Parallel Industrial NumErical

Applications and Portable

Libraries

pineapl project
PINEAPL Project
  • EU Framework IV Project in HPCN
  • Duration: 01/96 - 01/99
  • 31.4 man-year effort
  • Total budget: 3.4 m ECU (2.1 m funded)
  • Consortium of 10 partners

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

project objectives
Project Objectives
  • Development of a General-Purpose Library of Parallel Numerical Software
  • Suitable for Wide Range of Computationally Intensive Industrial Applications
  • Portable & Efficient Across a Wide Range of Parallel Machines
    • Distributed Multi-processor Machines
    • SMPs and Clusters of SMPs
    • Networks of Workstations and PCs
  • Incorporation into the NAG Parallel Library

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

pineapl consortium
PINEAPL Consortium

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

end users applications
BAe

Wing design: maximum lift/minimum drag

Electro-magnetics wave reflection of ducts

DHI

MIKE3: Estuarine and coastal hydraulics and oceanography - environmental simulation

Math-Tech

COSI: Oil reservoirsimulation

Piaggio

Engine simulation based on KIVA III: chemically reactant fluids with spray

Thomson

Nanometric recording devices

Beam propagation in diodes pumped rod lasers

End-users Applications

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

pineapl methodology
PINEAPL Methodology
  • Analyse industrial application codes
  • Utilise numerical expertise in participating research organisations to design and develop relevant PINEAPL Library software
  • Parallelize application codes using Library
  • Benchmark resulting parallel end-user codes

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

pineapl library software
PINEAPL Library Software
  • Sparse linear algebra
  • Mesh partitioning
  • PDE discretization and solution
  • (Un)Constrained non-linear optimization
  • FFTs: 1-, 2- and 3-dimensional
  • Banded and tridiagonal solvers
  • Dynamic load balancing
  • Utility and support routines

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

pineapl library rel 1
PINEAPL Library (rel 1)
  • Released February ‘98
  • Available free of charge for limited trial
  • Will be incorporated into Release 3 of NAG Parallel Library, due Q3 1998
  • Available for IBM SP2, SGI Origin 2000, Fujitsu AP3000, DEC Alpha SMPs, clusters of workstations, …

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

for further information
For Further information:

http://www.nag.co.uk/projects/PINEAPL.html

To Use the Library:

Available under a collaborative agreement.

Contact Mishi Derakhshan

[email protected]

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

nag parallel library rel 3
NAG Parallel Library(rel 3)
  • Quantum leap in coverage! (~ 120 new)
    • Sparse linear algebra (~ 75 new)
    • Dense linear algebra
    • FFTs
    • Optimization
    • Graph partitioning
  • Tested in real-world industrial applications

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

nag fortran smp library
NAG Fortran SMP Library

Release 1 of the NAG Library:

  • Launched at HPCN ‘97 (Vienna, April 1997)
  • Aimed (primarily) at cache-based SMP systems
  • Good one processor performance
  • Good scalability
  • Includes all NAG Fortran Library Interfaces
  • Over 50 routines benefit

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

nag fortran smp library24
NAG Fortran SMP Library

Parallelism on SMP systems:

  • Functional parallelism
  • Fork-Join Parallelism:

Parallel Region

Processes

Processes

Fork

Join

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

nag fortran smp library25
NAG Fortran SMP Library
  • Hide details of parallelism and system dependencies
  • Immediate migration of code between serial & SMP systems
  • Compiler directives:
    • single source codes for serial & SMP systems
    • rapid migration to other SMP platforms
  • Levels of parallel efficiency difficult to achieve otherwise
  • Easy access to high-performance for non-specialist

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

nag fortran smp library26
NAG Fortran SMP Library

SMP Library Parallelism coded as

Compiler Directives:

  • Coded comments ignored by Serial compiler
  • Different compilers have different directives grammar
  • Open MP: emerging standard for parallel compiler directives

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

compiler directives
Compiler Directives

A “Bestiary” or Compiler Directives:

C$PAR SGI SMP systems

C*KAP* Digital Alphaservers (KAP)

C$PAR Sun SMP Systems

*PDIR NEC SX-4

!$OMP IBM SMP systems

!$OMP, C$OMP, *$OMP Open MP

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

slide28

Release 1 Contents

  • Dense Linear Algebra
    • LU, QRandCholeskyfactorizations and related solvers
    • Symmetric eigenproblem:
      • reduction totridiagonal form
      • Compute Q
      • QR algorithm foreigenvalues and/or eigenvectors
  • FFT
    • Library routines interface vendor routines when these are available and perform well
    • New complex storage interfaces
    • New Nag auxiliary routines when vendors’ not available/suitable

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

slide29

NAG SMP Library, Rel. 1

SGI SCS Library v1.0

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

slide30

NAG SMP Library, Rel. 1

MKL Library 2.1

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

slide31

NAG SMP Library, Rel. 1

SGI SCS Library v1.0

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

nag parallel libraries
SMP

A library of numerical software including all the functionality of the NAG Fortran library,for SMP systems.

Provides a core set of high efficiency routines

Completely consistent interfaces and functionality with existing NAG products

NAG Parallel Libraries
  • Parallel
  • A library of numerical softwarefor distributed memory and shared memory systems
  • using either the PVM or MPI message-passing paradigms.
  • Future releases to benefit from development under PINEAPL project
  • Routines show good levels of performance and scalability

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

trends in hpc
Trends in HPC
  • Hardware
    • Seems to be a convergence on the clustered SMP - Hybrid machines.
    • Multiprocessor NT boxes also leading toward clustered SMP machine architecture.
  • Software
    • Trends harder to predict
      • MPI here to stay?
      • OpenMP
      • HPF possible player
      • C++, Java ?

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

issues for hybrid systems
Issues for hybrid systems
  • Software environment…
    • Open MP, MPI +
  • Algorithmic
    • Deepening Memory Hierarchy

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

ongoing investigation
Ongoing Investigation

Consider the marriage of

Parallel Library

+

SMP Library

Networks of NT multiprocessor machines.

Hybrid architectures

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

slide36
Assuming an MPI + compiler directives
  • Questions on variety of architectures:
    • what can we gain over simple MPI
    • how portable are the algorithms across platforms

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

providing an infrastructure
Providing an Infrastructure
  • Ease of use.
  • Distributed pieces of application.
  • Visualization of application.
  • Interaction with application.

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

slide38

IRIS Explorer

  • Visual Programming
  • Drag and Drop
  • Modular
  • Extensible
  • Multi-platform
  • Heterogeneous network

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

slide39

Modules

Modules are the basic building blocks.

Module has input and output ports for data transmission.

User designed interface to module.

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

slide40

Building an Application

  • Ports connected using mouse.
  • Connect to create a map - visual representation of program.
  • Can track execution through highlighted modules.
  • Parameters can be changed interactively.

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

slide41

The Librarian

  • Contains modules and maps.
  • Can be dragged onto map editor and ‘launched’.
  • Modules can be categorized in Library.
  • Modules on remote machines accessed by machine librarian.

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

slide42

The Module Builder

Extend IRIS Explorer. Interface to C, C++, Fortran.

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

projects using and expanding iris explorer
Projects using and Expanding IRIS Explorer
  • Julius - solving large PDE problems.
  • Decision - control systems.
  • Stable - Statistical environment.

Integration of library routines, Corba-compliance, collaborative development...

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

conclusion
Conclusion

NAG has some of the right pieces …

...figuring out how to put them together.

Anne E. Trefethen Workshop on Clusters, Blackberry Farm

ad