1 / 24

Lecture 4: Distributed-memory Computing with PVM/MPI

Lecture 4: Distributed-memory Computing with PVM/MPI. Cluster Computing. What is cluster computing Use a number of work stations or PCs to construct a virtual parallel computer. Comparing with parallel computers, cluster processing is cheap.

bona
Download Presentation

Lecture 4: Distributed-memory Computing with PVM/MPI

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Lecture 4: Distributed-memory Computing with PVM/MPI

  2. Cluster Computing • What is cluster computing • Use a number of work stations or PCs to construct a virtual parallel computer. • Comparing with parallel computers, cluster processing is cheap. • Performance is worse than parallel computers, but it can be largely improved by using high performance network.

  3. Global/Cloud Computing • What is global (GRID) computing Use all the resources on internet to construct a super distributed parallel virtual computer Super parallel computers Intranet + Internet High performance PC User PC cluster

  4. Implement of cluster processing • Standard software for cluster processing • PVM (Parallel Virtual Machine) • MPI (Message Passing Interface) • History of PVM • 1989PVM1.0 released by Oak Ridge National Laboratory. • 1991PVM2.0 released by University of Tennessee. • 1993PVM3.0 released. • 1994Published with free manual • PVM 3.4.3 released with C and Fortran. PVM: A Users’ Guide and Tutorial for Networked Parallel Computing http://www.netlib.org/pvm3/book/pvm-book.html

  5. Implement of cluster processing • History of MPI • 1993-1994Specification (MP-I) was decided by the • experts from 40 companies. • 1995 Specification of MPI-2 was decided. • 2001MPI-1.1 based implementation was released. It is used as default environment in many distributed parallel processing systems.

  6. Processor ID: 5 Processor ID: 3 receive(3) send(5, message) Common in PVM and MPI • Communication is based on 1 to 1 message passing. • They are designed for not only work stations and PCs but also parallel computers. • Corresponding to many OS and Languages. (OS: UNIX, Windows95/NT, Languages: C, Fortran, JAVA) • Free • Standard software in distributed and parallel computing.

  7. A virtual parallel computer pvmd pvmd (optima) (opty1) pvmd pvmd (opty2) (opty3) (Constructed by connected 4 computers ) Construction of PVM • Software of PVM • Daemon pvmd used for the communication between processors. • Console pvm used for constructing virtual parallel computer (xpvm is the console combined with GUI) • Library of functions such as pvm_send and pvm_receive for message sending and receiving. On any computer, one can use pvm or xpvm to construct a virtual parallel computer, and then make and execute programs.

  8. Start of PVM • (1) On any computer, use command pvm or xpvm to start pvmd. On optima, use pvm or xpvm to start pvmd .

  9. Start of PVM • (2)On console pvm add computers to construct a virtual parallel computer. (At the this time, pvmd is started in these computers.) On optima add opty1, opty2, opty3 to virtual parallel computer by pvmd.

  10. Execution of PVM • Program execution on PVM • (1)When the program which contains communication functions of pvm is executed, the specified programs will be executed by pvmd When program prog1 is executed on optima which contains communication functions, the process (in this case prog 1) will be started in all processors by pvmd.

  11. Execution of PVM • (2) Once the execution of a program started, communication and processing inside the program will be done automatically until the end. Processing is held with the communication between processors.

  12. Using PVM • Using command pvm • chen[11]% pvm • pvm> add opty1 opty2 opty3(add opty1, opty2, opty3 to the virtual computer) • 3 successful • HOST DTID • opty1 80000 • opty2 c0000 • opty3 100000 • pvm> conf(Show the configuration of the virtual parallel computer) • 4 hosts, 1 data format • HOST DTID ARCH SPEED DSIG • optima 40000 X86SOL2 1000 0x00408841 • opty1 80000 X86SOL2 1000 0x00408841 • opty2 c0000 X86SOL2 1000 0x00408841 • opty3 100000 X86SOL2 1000 0x00408841 • pvm> spawn edm4 (Execute program edm4 on the virtual computer) • 1 successful • t80001 • pvm> halt   (Halt of the virtual computer) • Terminated • chen[12]%

  13. XPVM Screen Shot

  14. Using XPVM Using xpvm (X application)

  15. PVM programming – Example 1 (Greeting) • 1. Example (hello.c, hello_other.c)

  16. PVM Programming – Example 1(Greeting) Start the process hello Hello, Dr. t80004 (Process hello_other is started on other computers by pvm_spawn. pvm_spawn Started process has its task ID.) (Process hello_other is started) Hello, Prof . t4000a. How are you? pvm_send (Send message by pvm_send) pvm_recv (Finish the process by pvm_exit) (Receive the message by pvm_recv) Fine, thank you, Dr. t80004 (Finish the process by pvm_exit)

  17. PVM programming – Example 2 (Calculate π) Equation of unit circle: Area of unit circle: πor

  18. PVM programming – Example 2 (Calculate π) Master Program (pi_dual.c) n=intervals/2; /* Interval for one subtask */ s=0;t=0.5; /* First subtask responsible for interval [0,0.5] */ pvm_initsend(0); /* Prepare for data sending */ pvm_pkint(&n, 1 ,1); /* Put the interval into the sending buffer */ pvm_pkdouble(&s, 1, 1); /* Put the value of s into the sending buffer */ pvm_pkdouble(&t, 1 ,1); /* Put the value of t into the sending buffer */ pvm_send(ctid[0], 1); /* Send the content of the buffer to the first subtask */ s=0.5;t=1; /* Secong subtask responsible for interval [0.5,1] */ pvm_initsend(0); /* Prepare for data sending */ pvm_pkint(&n, 1 ,1); /* Put the interval into the sending buffer */ pvm_pkdouble(&s, 1, 1); /* Put the value of s into the sending buffe */ pvm_pkdouble(&t, 1 ,1); /* Put the value of t into the sending buffe */ pvm_send(ctid[1], 1); /* Send the content of the buffer to the second subtask */ pvm_recv(-1, 1); /* Waiting for receiving the data from the subtasks, and put the data into the receiving buffer when received */ pvm_upkdouble(&result1,1,1); /* Put the received value (the result from the first subtask) into result1*/ pvm_recv(-1, 1); /* Waiting for receiving the data from the subtasks, and put the data into the receiving buffer when received */ pvm_upkdouble(&result2,1,1); /*Put the received value (the result from the second subtask)into result2*/ printf("Intervals=%d, PI=%1.15f",intervals,result1+result2); /* Show the result */ pvm_exit(); /* End */ exit(0); } /* Calculating PI by integral (Mater Program) */ #include<stdio.h>/* For standard I/O functions¡ */ #include"pvm3.h"/* For stand pvm functions */ main(argc,argv) intargc; char **argv; { int intervals; /* Number of intervals */ intctid[2]; /* task id (subtasks) */ int n; /* Number of intervals for one subtask */ doubles,t; /* Interval of tasks */ double result1,result2; /* Results of subtasks */ if(argc<=1){ /* Finish if no any parameter */ printf("Usage:pi_dual intervals\n"); pvm_exit(); exit(0); } sscanf(argv[1],"%d",&intervals); /* Get the first parameter and put it into intervals */ if(pvm_spawn("pi_slave",(char**) 0,0,"",2,ctid)!=2){ /* Start two subtasks for executing slave program pi_slave¡Save the subtask ids into arryctid */ printf("can't start pi_slave\n"); /* Fail in starting the subtasks */ pvm_exit(); exit(0); }

  19. PVM programming – Example 2 (Calculate π) Slave Program (pi_slave.c) /* Program for calculating PI by integral (slave program) */ #include<math.h>/* Using numerical fuctions (sqrt, etc.) */ #include"pvm3.h"/* Using pvm functions */ main() { intptid; /* Master task id */ int n; /* Number of intercals */ doubles,t; /* Interval of integral */ inti; /* Variable for loop */ double h; /* Width of one interval */ double sum=0; /* Accumulated value of the area. Initial value=0 */ double x; /* x coordinate used at present */ ptid= pvm_parent(); /* put master task id into ptid */ pvm_recv(ptid, 1); /* Wating for receiving the data from master task, and put the data into the receiving buffer when received */ pvm_upkint(&n, 1, 1); /*Put the interval assigned for one task into n */ pvm_upkdouble(&s, 1 ,1); /*Put the left endpoint of the interval to s*/ pvm_upkdouble(&t, 1 ,1); /* Put the right endpoint into t */ h=(t-s)/n; /* Width of small intervals */ for (i=0;i<n;++i){ /* Repeat the loop n times */ x=(i+0.5)*h+s; /* Midpoint of x coordinate */ sum+=(4*sqrt(1-x*x))*h; /* Calculate the area of one small interval and add it to sum. */ } pvm_initsend(0); /* Prepare for sending data */ pvm_pkdouble(&sum, 1, 1); /* Put sum into the sending buffer */ pvm_send(ptid, 1); /* Send the content of the sending buffer into the master process */ pvm_exit(); /* End */ exit(0); }

  20. MPI programming – Example 1(Greeting) slave master

  21. MPI programming – Example 2 (Calculate π) #include "mpi.h" #include <stdio.h> #include <math.h> int main( intargc, char *argv[] ) { int n, myid, numprocs, i; double PI25DT = 3.141592653589793238462643; double mypi, pi, h, sum, x; MPI_Init(&argc,&argv); MPI_Comm_size(MPI_COMM_WORLD,&numprocs); MPI_Comm_rank(MPI_COMM_WORLD,&myid); while (1) { if (myid == 0) { printf("Enter the number of intervals: (0 quits) "); scanf("%d",&n); } MPI_Bcast(&n, 1, MPI_INT, 0, MPI_COMM_WORLD); if (n == 0) break; else { h = 1.0 / (double) n; sum = 0.0; for (i = myid + 1; i <= n; i += numprocs) { x = h * ((double)i - 0.5); sum += (4.0 / (1.0 + x*x)); } mypi = h * sum; MPI_Reduce(&mypi, &pi, 1, MPI_DOUBLE, MPI_SUM, 0, MPI_COMM_WORLD); if (myid == 0) printf("pi is approximately %.16f, Error is %.16f\n", pi, fabs(pi - PI25DT)); } } MPI_Finalize(); return 0; } one task reduction function for parallel summation

  22. MPI programming – Reduction

  23. PVM? MPI? • PVM is easy to use, especially on a network of workstations. Its message passing API is relatively simple • MPI is a standard, has a steeper learning curve and doesn’t have a standard way to start tasks MPICH does have an “mpirun” command • If building a new scalable, production code, should use MPI (widely supported now) • If experimenting with message passing, are interested in dynamics, use PVM. • MPI has a very rich messaging interface and designed for efficiency http://www-unix.mcs.anl.gov/mpi • PVM has a simple messaging interface + Process control, Interoperability, Dynamics: http://www.epm.ornl.gov/pvm • Perform comparably when on Ethernet • MPI outperforms when on MPP • Both are still popular, but MPI is an accepted community standard with many support chains.

  24. (1) Read the PVM program pi_dual.c and pi_slave.c which are used for calculating. Compile pi_dual.c and pi_slave.c. Learn to use xpvm. (2) Revise program pi_dual.c to pi_single.c such that it uses only one pi_slave task. Find N (the number of the divided intervals) such that the running time of pi_single is 30 sec, 60 sec, 90 sec, and 120 sec, respectively. (3) Compare the running time of pi_dual and pi_single using the different values of N obtained in (2). Notice that the ideal speed-up rate is (4) Revise program pi_dual.c to pi_multi.c such that it can use any number of slave tasks. For example, Pvm> spawn -> pi_multi 200000000 4 starts 4 slave tasks which do calculation in intervals [0,0.25],[0.25,0.5],[0.5,0.75],[0.75,1], respectively, where each interval is divided to 50000000 smaller intervals. • Exercise (5) For each value of N obtained in (2), get the running time pi_multi when the number of slave tasks is 1,2,4,8,16, respectively, find the speed-up rate (notice that the ideal rate is, investigate the change in the speed-up rates, and discuss the reasons. (6) Consider how to balance the work load among the host computers. (Hint: according to the work load of each computer, the time needed for running pi_slave is different. Assign more slave tasks to the computers which has lower work load. Design the PVM program to realize your idea, and discuss how much the running time and the speed-up rate are improved. (7) Change the above PVM programs into the MPI programs.

More Related