High performance computing systems and enabling platforms
Download
1 / 39

High Performance Computing Systems and Enabling Platforms - PowerPoint PPT Presentation


  • 106 Views
  • Uploaded on

Master Program (Laurea Magistrale) in Computer Science and Networking Academic Year 2009-2010. High Performance Computing Systems and Enabling Platforms. Marco Vanneschi Department of Computer Science, University of Pisa.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' High Performance Computing Systems and Enabling Platforms' - krista


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
High performance computing systems and enabling platforms

Master Program (Laurea Magistrale) in Computer Science and NetworkingAcademicYear 2009-2010

High Performance ComputingSystemsand EnablingPlatforms

Marco Vanneschi

Departmentof Computer Science, Universityof Pisa


Master Program (Laurea Magistrale) in Computer Science and Networking

High Performance ComputingSystems and EnablingPlatforms

Marco Vanneschi

CourseIntroduction


My activity
Myactivity

  • Research area

    • Computer Architecture

    • Parallel and Distributed Processing, High Performance Computing

    • ParallelProgrammingModels and Tools

    • Programmabilityofvarious HPC platforms

      • Multiprocessor, Cluster, GridComputing, Multi-core, Pervasive Computing

    • Coordinationof some National and EuropeanProjects (basicresearch and industrial research)

  • Researchgroup

    • Co-leadedwith Prof. Marco Danelutto

    • LaboratoryofParallelArchitectures

  • Strong relationshipresearch - teaching

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


This course acronym spa
Thiscourse (acronym: SPA)

My Personal Page:www.di.unipi.it/vanneschsection: Teaching

Link in DidaWikitomy Personal Page: http://www.cli.di.unipi.it/doku/doku.php/magistraleinformaticanetworking/spa/start

  • FundamentalcourseofLaurea Magistrale in Computer Science and Networking, 1st Year

  • In common withLaurea Magistrale in Informatica

    • complementarycourse (studyplan in DistributedSystems)

  • ASE (vecchio ordinamento, 9 CFU): 6 CFU di SPA + 3 CFU di integrazione (vedi in seguito)

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


SPA

6 CFU = 48 hours

(12 weeks)

CREDIT DEFINITION:

1 CFU = 25 hours =

8 hours for lectures / class activities (lab, practical, etc)

+

17 hours for individual study

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Contents
Contents

  • Objectives, motivations, approach

  • An informalpresentationof some concepts and technologies

  • Background and prerequisites

  • Courseprogram

  • Course material/notes

  • Exammodality and workingapproach

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Contents1
Contents

  • Objectives, motivations, approach

  • An informalpresentationof some concepts and technologies

  • Background and prerequisites

  • Courseprogram

  • Course material/notes

  • Exammodality and workingapproach

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Course objectives
Courseobjectives

  • Provide a solidknowledgeframeworkofconcepts and techniques in high-performance computer architecture

    • Organization and structureofenablingplatformsbased on parallelarchitectures

    • Supporttoparallelprogrammingmodels and software developmenttools

    • Performance evaluation (costmodels)

  • Methodologyforstudyingexisting and future systems

  • Technology: state-of-the-art and trends

    • Parallelprocessors

    • Multiprocessors

    • Multicore / manycore / … / GPU

    • Sharedvsdistributedmemoryarchitectures

    • Programmingmodels and theirsupport

    • General-purposevsdedicatedplatforms

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Motivations
Motivations

  • Basically, the samemotivationdiscussed in “DistributedSystems: Paradigms and Models” (Prof. Marco Danelutto):

    • evolutionof computer technologytowardsparallelism and HPC

      • Multi/manycore

      • Large cluster, cloud, …

      • Heterogeneouslarge-scaleenabling ICT platforms

      • Embedding

  • Increasingmaturitywithrespectto “hardware-software relationship”

    • BothTechnologyPushand Technology Pull

  • Language-drivenarchitecturalapproaches

  • Concurrency and parallelismasfirst-classcitizens in applicationdevelopment

  • In our Master: HPC is a fundamentalmethodology and technologyforintegrated ICT infrastructures and applications

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Course approach a computer science approach
Courseapproach: a Computer Science approach

  • The concept of Enabling Platform: strong relationship and integration between architectures and applications

    • Computing architectures are NOT “boxes and wires”

methodological

knowledge

technological

knowledge

This approach does not imply that concrete technologies (of current and future) computing platforms are neglected – far from it: our goal is to fully understand and to utilize existing architectures at best, and even to define new ones.

  • Computer Science approach

    • Computingarchitecturehasitsownconcepts, principles, models, and techniques

    • Conceptualframework in common with the otherdisciplinesof Computer Science:

      • Programminglanguages, algorithms, computability and complexity, …

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Contents2
Contents

  • Objectives, motivations, approach

  • An informalpresentationof some concepts and technologies

  • Background and prerequisites

  • Courseprogram

  • Course material/notes

  • Exammodality and workingapproach

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Hpc enabling platforms

Main Memory and Cache Levels

CPU

CPU

CPU

CPU

CPU

HPC enablingplatforms

  • Sharedmemorymultiprocessors

    • Varioustypes (SMP, NUMA, …)

Main Memory

and/or Cache levels

. . .

  • From simple to very sophisticated memory organizations

  • Impact on the programming model and/or process/threads run-time support

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Hpc enabling platforms shared and distributed memory architectures

2100

2100

2100

2100

2100

2100

2100

2100

CPU

CPU

CPU

HPC enablingplatforms: shared and distributedmemoryarchitectures

InstructionLevelParallelism CPU (pipeline, superscalar,

multithreading, …)

M

CPU

. . .

Shared memory multiprocessor

“Limited degree” Interconnection Network (“one-to-few”)

Distributed memory multicomputer: PC cluster, Farm, Data Centre, …

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Cpu technology evolution and multicore

L2 cache

L1 cache

L1 cache

P

P

P

P

P

P

P

P

CPU chip – SMP multiprocessor

CPU technologyevolution and multicore

Multiprocessor on single chip

  • “Dramatic revolution” for ITC industry: programmability issues

  • Computer providers support : from sequential programming to parallel programming

  • Also: NETWORK PROCESSORS with multicore technology

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Multicore technology examples
Multicoretechnologyexamples

SUN

Niagara 3

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Multicore technology examples1
Multicoretechnologyexamples

IBM

Power 7

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Multicore technology examples2
Multicoretechnologyexamples

IBM

CELL BE

(out of production …)

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Hpc enabling platforms1

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

2100

Administrative Domain

Adm. domain

Adm.domain

Linux / Pentium

Power PC / MacOS

Seemless

global system view

SUN / Solaris

HPC enablingplatforms

Heterogeneous Clusters

Homogeneous Clustes,in general with multiprocessor/multicore nodes (SMP, NUMA, …)

Virtual Private Networks: Farms, Data Centres

Large Scale Platforms (LAN, MAN, WAN):Grids, Clouds, …

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Large scale platforms grids and much more

Added value: Quality of Service (QoS)

Distributed/Web Computing: Objects / Component Software Technology

High-performance Computing, Cluster Computing

Knowledge Management and Data Intensive Computing

Cooperation and Virtual Organizations, Pervasive Computing

Large scale platforms (Grids and much more)

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Example of heterogeneous distributed hpc platform pervasive grid

Cluster

Cluster

Data & Knowledge

Server

Workstation

Workstation

Workstation

Workstation

Exampleofheterogeneousdistributed HPC platform: Pervasive Grid

An integratedsystem composedofcentralservers and services, fixed and mobile decentralizednodes, variouskindofnetworks.

A distributedapplication, e.g. emergency management,mustbeableto exploit all the processing andcommunicationresources at best.

PDA

PDA

PDA

PDA

PDA

PDA

PDA

Variouskindoffixed, mobile, and

ad-hocnetworks

Meteo services

GIS services

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Example of pervasive grid application
Exampleof Pervasive Gridapplication

Flood management

Data- and computation-intensiveactivities: OFF-LINE and REAL-TIME

Water level, speed, soil status, …

HydrologicalModel: floodwave

Data Mining

Along the river , …

Visualization

Post-processing(non trivial)

Precipitation in time and space,

fromsatellites, meteo radars, raingauges, …

Autorities, supervisors, observers, rescuers, police, firemens, …

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Off line vs real time adaptive processing

Cluster

Cluster

Data & Knowledge

Server

Workstation

Workstation

Workstation

Workstation

Off-line vs real-time adaptive processing

The tipicaloff-line, routinelytasks involve centralservers and some predefinednetworks. Mobile remote devices are usedmainlyforcommunication and visualization.

In emergencysituations, tasks can bere-allocatedto remote nodes/devices and mobile networks in real-time (e.g. centralresources are disconnected or communicationisinefficient).

Are remote nodes/devices and networksabletoprocesshigh-performancetaks?

PDA

PDA

PDA

PDA

PDA

PDA

PDA

Variouskindoffixed, mobile, and

ad-hocnetworks

Meteo services

GIS services

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


The impact of multicore on next generation dedicated and mobile technology

M

P

P

P

P

P

P

P

P

The impact ofmulticore on next-generationdedicated and mobile technology

PDA

Embedding into mobile and /or wearable intelligent devices

PDA

PDA

PDA

Data & Knowledge

Server

Cluster

On-chip Multiprocessor

High-performance computing on a distributed collection of “simple” remote nodes/devices

is feasible (and can be very efficient)

PDA

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Contents3
Contents

  • Objectives, motivations, approach

  • An informalpresentationof some concepts and technologies

  • Background and prerequisites

  • Courseprogram

  • Course material/notes

  • Exammodality and workingapproach

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Basic background and prerequisites
Basic background and prerequisites

  • An undergraduate-levelcourse on structured computer architecture

    • Firmwarelevelstructuring

    • Assemblerlevel, CPU architecture, compiling

    • Memoryhierarchies and caching

    • Interrupt handling, exceptionhandling

    • Processlevel, addressingspace, low levelscheduling, interprocesscommunication

    • Input/Output processing

  • Fewbooksadopt a structuredapproach:

    • Tanenbaum: in principle, some partsonly

    • Patterson-Hennessy: mainlydescriptionofexistingtechnologies (fewconcepts)

  • In Pisa: course “Computer Architecture”

    • Book: M. Vanneschi, “Architettura degli Elaboratori”, PLUS (Pisa University Press), 2009 – in Italian!

  • Some initiallectureswillreview the basicconceptsof the structuredapproach

    • Students are stronglyinvitedtoattendthis part in a verycriticalmanner

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Background from other courses of mcsn
Background fromothercoursesof MCSN

  • Course by Prof. Marco Danelutto (Distributed Systems: Paradigms and Models)

    • Structures of parallel computations

    • Performance measures and cost models

      • Service time / bandwidth, latency, completion time, efficiency, scalability

    • Basic mechanisms for process cooperation (messages, shared variables)

    • Parallelism forms / paradigms, structured parallelism

      • Stream-parallel pipeline

      • Stream-parallel farm

      • Data-flow

      • Data-parallel map, reduce, parallel prefix

      • Data-parallel with stencils

    • Client-server computations

    • Impact of service-time and latency on client service-time

  • Course of Advanced Programming

  • Basic elements of Queueing Theory

  • Basic elements of Networking

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Contents4
Contents

  • Objectives, motivations, approach

  • An informalpresentationof some concepts and technologies

  • Background and prerequisites

  • Courseprogram

  • Course material/notes

  • Exammodality and workingapproach

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Course program
CourseProgram

PART 1( 1/4)

  • Prerequisites revisited

    • Firmware structuring; processors, memory hierarchies and caching; assembler level and compiler optimizations, performance parameters; process cooperation and implementation

  • Run-time support to concurrency mechanisms

    • Structured interpretation of process communication and sharing

  • Instruction level parallelism

    • Elements of pipeline and superscalar CPUs, cost models, compiler optimizations

      PART 2( 3/4)

  • Shared memory architectures

    • SMP, NUMA, …, interconnection networks, support to concurrency mechanisms, cost models, static and dynamic optimizations, parallel application benchmarks

  • Distributed memory architectures

    • Cluster, MPP, …, interconnection networks, support to concurrency mechanisms, cost models, static and dynamic optimizations, parallel application benchmarks

  • Multicore architectures

    • Current status and trends of single-chip shared/distributed memory architectures

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Further mcsn courses on these subjects
Further MCSN courses on thesesubjects

  • ComplementsofDistributedEnablingPlatforms (CAP)

  • ProgrammingToolsforParallel and DistributedSystems (SPD)

    • For the currentyearonly (free-choiceexam): mergedinto the samecourse (formally: SPD)

      • Grid and Cloud,

      • DistibutedOperatingSystems,

      • Tools and LibrariesforParallel and DistributedMachines (MPI and otherstandards or commercial products; ASSIST–Universityof Pisa and possiblyotherresearchtools)

      • Virtualization,

      • Scheduling.

    • Nextyear (studyplan): 2 distinctcourses (CAP: 6 CFU, SPD: 9 CFU)

  • Parallel and DistributedAlgorithmsNextyear

    Toincrease the knowledgeof some notableapplicationparadigms:

  • NumericalTechniques and Applications (TNA)

  • Network OptimizationMethodsNextyear

  • Data MiningTechniquesNextyear

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Contents5
Contents

  • Objectives, motivations, approach

  • An informalpresentationof some concepts and technologies

  • Background and prerequisites

  • Courseprogram

  • Course material/notes

  • Exammodality and workingapproach

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


The lectures
The lectures

  • Slides and blackboard

  • Slides for

    • (part of) course material

    • lecture outline

  • Blackboard

  •  for each slide, where necessary or convenient: further explanation / discussion using the blackboard

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Course material
Course Material

  • Mypage:

    • www.di.unipi.it/vanneschsection: Teaching (*)

    • Link in DidaWiki: http://www.cli.di.unipi.it/doku/doku.php/magistraleinformaticanetworking/spa/start

  • Lecture Notes

    • Slides (*)

    • Documents (*)

  • Papers and selected book chapters

  • M. Vanneschi, “Architettura degli Elaboratori”, PLUS, 2009

    • Part IV

    • English version: nextyear

    • Some partswillbytranslated in English during the course (*)

  • ReferenceBooks

    • D.A.Patterson, J.H.Hennessy, “Computer Organization and Design: the Hardware/Software Interface”, Morgan Kaufman Publishers Inc.

    • D.E.Culler, J.P.Singh, A. Gupta, “Parallel Computer Architecture: a Hardware/Software Approach”, Morgan Kaufman Publishers Inc.

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Contents6
Contents

  • Objectives, motivations, approach

  • An informalpresentationof some concepts and technologies

  • Background and prerequisites

  • Courseprogram

  • Course material/notes

  • Exammodality and workingapproach

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


Exam modality
Exammodality

Forallstudents:

Written test + oral test (in English or in Italian)

Written test: explanation/discussionofconcepts and techniquesof the course, notnecessarilyfocused on smallexercizes. Emphasiswillbe put on the knowledgeofmethodologies and theirapplication, aswellas on the synthesiscapability and on the student’s abilitytoestablish the properrelationshipsbetween the variouspartsof the course.

Optional: a report on a specifictopic

  • individualwritten report, maximum 2 persons

  • No intermediate tests

  • Report tobesubmitted a certaintime in advancewrt the exam date.

  • ASE: see a subsequent slide.

  • Registrationto the exam on the Official Site of Corso di Laurea: http://compass2.di.unipi.it/didattica, section Laurea Magistrale in Informatica e Netwoking, subsection “orari”

  • MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


    Exam modality1
    Exammodality

    Report

    • Some literature material (e.g. one/some papers) isassignedto the student

      • existingparallelmachines / multicore, or existingprojects,

      • specifictechniques and/or technologies on topicsof interest .

    • The assigned material mustbestudied and interpretedaccordingto the coursecontents, methodology and approach

    • The report mustbewritten in a didactic style, asitwere a book chapterforstudents (“student-proof”)

      • “ ifanauthorisnotabletoexplain a certainthing in anunderstandable and complete manner, thencertainlysuchthingisnotclearto the authorhimself ”

    • Literatureassignedduring the first 2-3 weeksof the course

    MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


    Architetture parallele e distribuite ase
    Architetture Parallele e Distribuite (ASE)

    Esame di ASE (9 CFU), laurea specialistica vecchio ordinamento:

    • SPA (6 CFU)

    • + integrazione 3 CFU sulla parte delle metodologie di parallelizzazione (Libro Vanneschi, Cap. X)

    • Modalità di esame: scritto tradizionale e orale

    • All’atto dell’iscrizione: indicare ASE

    MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


    Working approach
    Workingapproach

    • As in any other course, it is fundamental to acquire skills and capabilities in concepts and principles, besides knowing the technologies.

    • Critical aptitude must be properly developed.

    • Interaction with the teacher is strongly encouraged

      • Questions during the lectures

      • Question time (“orario di ricevimento”) (in Italian for Italians)

        • Wednesday, 14:30 – 17:30, in my room

        • or by appointment in case of collision with other courses.

    MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


    Question Time

    14:30 -17:30, my room (Dept)

    or by appointment

    in case the student attends the Wednsday afternoon lectures

    MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


    Good Luck !

    MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms


    ad