1 / 21

Parallel and Distributed Computing: Foundations and Applications

Learn the fundamentals of parallel and distributed computing, including socket programming, thread programming, cluster architecture, parallel programming models, and cloud computing concepts. Gain practical skills in writing multi-threaded servers and clients, performance evaluation on real clusters, developing grid computing systems, and more.

chamberlains
Download Presentation

Parallel and Distributed Computing: Foundations and Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 678 Topics Covered (1) • Part A: Foundation • Socket Programming • Thread Programming • Elements of Parallel Computing • Part B: Cluster Computing • Elements of Cluster Computing • Cluster Architecture and Components • Single System Image: Concepts and Levels • Parallel Programming Models and Paradigms • MPI Programming

  2. 678 Topics Covered (2) • Part C: Grid and Cloud Computing • Elements of Grid Computing • Grid Resource Management and Grid Economy • Enterprise Grids and the Aneka Middleware • Nimrod-G Grid Resource Broker • Economic Scheduling Algorithms • Gridbus Broker for Data-Intensive Computing • Globus Toolkit • Cloud Computing and Aneka

  3. Exposure to Industrial Developments • Unique “feature” of this subject: • CLOUDS Lab supported speakers • Industry Speakers (one or more) • Microsoft (HPC Specialist) • Sun (Senior Director of Grid Computing) • Amazon (Architect, Amazon Cloud Computing) • Microsoft HPC Server • Cluster Computing • Commercial Applications • Sun Grid Engine (SGE) • Seamless scaling from Clusters to Grids • Story on idea successful commercialisation • Amazon EC2/S3 • Computing as utility (pay-as-you-go services for IT) • AWS Marketplace

  4. 678 Practical Skills Acquired (1) • Part A: Foundation • Socket Programming • Thread Programming • How to write multi-threaded servers and clients • Part B: Cluster Computing • Parallel Programming: • MPI Programming • Coordination of multiple processes and load balancing • Performance Evaluation on real Clusters

  5. 678 Practical Skills Acquired (2) • Part C: Grid/Cloud Computing • Report Writing • Team-based projects • Medium scale software development • Developing skills in “Grid” computing technology: • Writing a simple Grid resource management system • Client with coordination ability • QoS-based scheduling • User-Interface design for network applications • Cloud Computing (Concepts and Industry Trends)

  6. Quick Revision

  7. 678 Topics Covered (1) • Part A: Foundation • Socket Programming • Thread Programming • Elements of Parallel Computing

  8. Client Java Sockets ServerSocket(1254) • server Output/write stream Input/read stream Socket(“128.250.25.158”, 1254) It can be host_name like “mandroo.cs.mu.oz.au”

  9. Threads Interface Microkernel Multi-Processor Computing System . . P P P P P P P Processor Process Thread Parallel Computing Elements Applications Programming paradigms Operating System Hardware

  10. 678 Topics Covered (1) • Part B: Cluster Computing • Elements of Cluster Computing • Cluster Architecture and Components • Single System Image: Concepts and Levels • Parallel Programming Models and Paradigms • MPI Programming

  11. PC/Workstation PC/Workstation PC/Workstation PC/Workstation Communications Software Communications Software Communications Software Communications Software Network Interface Hardware Network Interface Hardware Network Interface Hardware Network Interface Hardware Cluster Architecture Parallel Applications Parallel Applications Parallel Applications Sequential Applications Sequential Applications Sequential Applications Parallel Programming Environment Cluster Middleware (Single System Image and Availability Infrastructure) Cluster Interconnection Network/Switch

  12. What is Single System Image (SSI)? • SSI is the illusion, created by software or hardware, that presents a collection of computing resources as one, more whole resource. • In other words, it the property of a system that hides the heterogeneous and distributed nature of the available resources and presents them to users and applications as a single unified computing resource. • SSI makes the cluster appear like a single machine to the user, to applications, and to the network.

  13. Application and Subsystem Level Operating System Kernel Level Hardware Level SSI Levels • SSI levels of abstractions:

  14. Levels of Parallelism Code-Granularity Code Item Large grain (task level) Program Medium grain (control level) Function (thread) Fine grain (data level) Loop (Compiler) Very fine grain (multiple issue) With hardware Task i-l Taski Task i+1 PVM/MPI func1 ( ) { .... .... } func2 ( ) { .... .... } func3 ( ) { .... .... } Threads a ( 0 ) =.. b ( 0 ) =.. a ( 1 )=.. b ( 1 )=.. a ( 2 )=.. b ( 2 )=.. Compilers + x Load CPU

  15. Methodical Design or Stages of Parallel Programs • Partitioning • Decomposition of computational activities and the data into small tasks – there exist number of paradigms – e.g. master worker, pipeline, divide and conquer, SPMD, and speculation. • Communication • Flow of information and coordination among tasks that are created in the portioning stage. • Agglomeration • Tasks and communication structure created in the above stages are evaluated for performance and implementation cost. Tasks may be grouped into larger tasks to improve communication. Individual communications can be bundled. • Mapping / Scheduling • Assigning tasks to processors such that job completion time is minimized and resource utilization is maximized. Even the cost of computation can be minimized based on QoS requirements.

  16. 678 Topics Covered (2) • Part C: Grid and Cloud Computing • Elements of Grid Computing • Grid Resource Management and Grid Economy • Enterprise Grids and the Aneka Middleware • Nimrod-G Grid Resource Broker • Economic Scheduling Algorithms • Gridbus Broker for Data-Intensive Computing • Globus Toolkit • Cloud Computing (Concepts and Industry Trends)

  17. 2100 2100 2100 2100 2100 2100 2100 2100 Grid Resources and Scheduling User Application Grid Resource Broker Grid Information Service Local Resource Manager Local Resource Manager Local Resource Manager Single CPU (Time Shared Allocation) SMP (Time Shared Allocation) Clusters (Space Shared Allocation)

  18. workload Gridbus User Console/Portal/Application Interface App, T, $, Optimization Preference Gridbus Broker Gridbus Farming Engine Schedule Advisor Trading Manager RecordKeeper Grid Dispatcher Grid Explorer TM TS $ GE GIS, NWS Core Middleware Grid Info Server RM & TS G $ Data Catalog Data Node C $ U G Unicore enabled node. Globus enabled node. L A

  19. Deadline (D) and Budget (B) Constrained Scheduling Algorithms

  20. Convergence of Paradigms/Communities  Realisation of Leonard’s “computer utilities” Vision ? } • Web • Data Centres • Utility Computing • Service Computing • Grid Computing • P2P Computing • Cloud Computing • Market-Oriented Computing • … + • -Ubiquitous access • -Reliability • Scalability • Autonomic • Dynamic discovery &composiability • -QoS • -SLA • - … • Trillion $ business • Who will own it? Paradigms Attributes/Capabilities

  21. SES (Subject Experience Survey) • Q4, subject is well taught - can be interpreted as: I feel good about what I learnt, genuine effort by teachers  5. Strongly agree : 4. Agree : 3. Neutral : 2. Disagree : 1. Strongly disagree :

More Related