1 / 31

Embarrassingly Parallel Computations

Embarrassingly Parallel Computations. Chapter 3. 1. Ideal Parallel Computation. In this chapter, we will consider applications where there is minimal interaction between the slave processes. Slaves may be statically assigned or dynamically assigned.

dgean
Download Presentation

Embarrassingly Parallel Computations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Embarrassingly Parallel Computations Chapter 3

  2. 1. Ideal Parallel Computation • In this chapter, we will consider applications where there is minimal interaction between the slave processes. • Slaves may be statically assigned or dynamically assigned. • Load balancing techniques can offer improved performance. • We will introduce simple load balancing here • Chapter 7 will cover load balancing when the slaves must interact.

  3. 1. Ideal Parallel Computation • Parallelizing these should be obvious and requires no special techniques or algorithms. • A truly embarrassingly parallel computation requires no communication. (SPMD Model)

  4. 1. Ideal Parallel Computation • Nearly embarrassingly parallel computations • Require results to be distributed and collected in some way. • A common approach is the Master-Slave organization.

  5. 2. Embarrassingly Parallel Examples • 2.1 Geometrical Transformation of Images • The most basic way to store a computer image is a pixmap (each pixel is stored in a 2D array) • Geometrical transformations require mathematical operations to be performed on the coordinates of each pixel without affecting its value.

  6. 2.1 Geometrical Transformation of Images • There are several different transformations • Shifting • Object is shifted Dx in the x dimension and Dy in the y dimension: • x’=x+ Dx • y’=y+ Dy • Where x and y are the original and x’ and y’ are the new coordinates. • Scaling • Object scaled by factor Sx in the x-direction and Sy in the y-direction • x’=xSx • y’=ySy

  7. 2.1 Geometrical Transformation of Images • Rotation • Object rotated through an angle q about the origin of the coordinate system: • x’=x cosq + y sinq • y’=-x sinq + y cosq • Clipping • Deletes from the displayed picture those points outside the defined area.

  8. 2.1 Geometrical Transformation of Images • The input data is the pixmap (bitmap) • Typically held in a file, then copied into an array. • The master is concerned with dividing up the array among the processors. • There are two methods: • Blocks • Stripes

  9. 2.1 Geometrical Transformation of Images

  10. 2.1 Geometrical Transformation of Images

  11. 2.1 Geometrical Transformation of Images: Master – pseudo code for (i = 0, row = 0; i < 48; i++, row = row + 10) /* for each process*/ send(row, Pi); /* send row no.*/ for (i = 0; i < 480; i++) /* initialize temp */ for (j = 0; j < 640; j++) temp_map[i][j] = 0; for (i = 0; i < (640 * 480); i++) { /* for each pixel */ recv(oldrow,oldcol,newrow,newcol, PANY); /* accept new coords */ if !((newrow < 0)||(newrow >= 480)||(newcol < 0)||(newcol >= 640)) temp_map[newrow][newcol]=map[oldrow][oldcol]; } for (i = 0; i < 480; i++) /* update bitmap */ for (j = 0; j < 640; j++) map[i][j] = temp_map[i][j];

  12. 2.1 Geometrical Transformation of Images: Slave – pseudo code recv(row, Pmaster); /* receive row no. */ for (oldrow = row; oldrow < (row + 10); oldrow++) for (oldcol = 0; oldcol < 640; oldcol++) {/* transform coords */ newrow = oldrow + delta_x; /* shift in x direction */ newcol = oldcol + delta_y; /* shift in y direction */ send(oldrow,oldcol,newrow,newcol, Pmaster);/* coords to master */ }

  13. 2.1 Geometrical Transformation of Images: Analysis • Sequential: If each pixel required 1 time step then • ts = n2 • So, time complexity is O(n2) • Recall that for parallel • tp = tcomp + tcomm • tcomm = tstartup + mtdata =O(m) • tcomp = 2(n2/p) = O(n2/p) • tp = O(n2) • What’s the problem?

  14. 2.1 Geometrical Transformation of Images: Analysis • The constants for the communication (for parallel) far exceed the constants in the computation (for sequential) • Makes it relatively impractical • This problem is probably better suited for a shared memory system.

  15. 2.2 Mandelbrot Set • Def: set of points in the plane that are quasi-stable when computed by iterating the function: • zk+1 = zk2+c • Note: • z is complex, and c is complex. • The initial value for z is 0 • Iterations continue until • the magnitude of z is greater than 2 • or the number of iterations reaches some limit

  16. 2.2 Mandelbrot Set

  17. 2.2 Mandelbrot Set: Sequential int cal_pixel(complex c) { int count, max; complex z; float temp, lengthsq; max = 256; z.real = 0; z.imag = 0; count = 0; /* number of iterations */ do { temp = z.real * z.real - z.imag * z.imag + c.real; z.imag = 2 * z.real * z.imag + c.imag; z.real = temp; lengthsq = z.real * z.real + z.imag * z.imag; count++; } while ((lengthsq < 4.0) && (count < max)); return count; }

  18. 2.2 Mandelbrot Set: Parallel • Static Task Assignment – Master for (i = 0, row = 0; i < 48; i++, row = row + 10) /* for each process*/ send(&row, Pi); /* send row no.*/ for (i = 0; i < (480 * 640); i++) { /* from processes, any order */ recv(&c, &color, PANY); /*receive coords & colors */ display(c, color); /* display pixel on screen */ }

  19. 2.2 Mandelbrot Set: Slave recv(&row, Pmaster); /* receive row no. */ for (x = 0; x < disp_width; x++) /* screen coordinates x and y */ for (y = row; y < (row + 10); y++) { c.real = min_real + ((float) x * scale_real); c.imag = min_imag + ((float) y * scale_imag); color = cal_pixel(c); send(&c, &color, Pmaster);/* send coords, color to master */ }

  20. 2.2 Mandelbrot Set • Dynamic Task Assignment • Work Pool/Processor Farms

  21. Master count = 0; /* counter for termination*/ row = 0; /* row being sent */ for (k = 0; k < procno; k++) { /* assuming procno<disp_height */ send(&row, Pk, data_tag); /* send initial row to process */ count++; /* count rows sent */ row++; /* next row */ } do { recv (&slave, &r, color, PANY, result_tag); count--; /* reduce count as rows received */ if (row < disp_height) { send (&row, Pslave, data_tag); /* send next row */ row++; /* next row */ count++; } else send (&row, Pslave, terminator_tag); /* terminate */ rows_recv++; display (r, color); /* display row */ } while (count > 0);

  22. 2.2 Mandelbrot Set: Slave recv(y, Pmaster, ANYTAG, source_tag); /* receive 1st row to compute */ while (source_tag == data_tag) { c.imag = imag_min + ((float) y * scale_imag); for (x = 0; x < disp_width; x++) { /* compute row colors */ c.real = real_min + ((float) x * scale_real); color[x] = cal_pixel(c); } send(&i, &y, color, Pmaster, result_tag); /* row colors to master */ recv(y, Pmaster, source_tag); /* receive next row */ };

  23. 2.2 Mandelbrot Set

  24. 2.2 Mandelbrot Set: Analysis • Sequential • ts= max x n = O(n) • Parallel • Phase 1: Communication Out • tcomm=s(tstartup+tdata) • It could be possible to use a scatter routine here which would cut the number of startup times. • Phase 2: Computation • (max x n)/s • Phase 3: Communication Back in • tcomm2= (n/s)(tstartup+tdata) • Overall • tp <= (max x n)/s + (n/s + s)(tstartup+tdata)

  25. 2.3 Monte Carlo Methods • The basis of Monte Carlo methods is the use of random selections in calculations. • Example – Calculate p • A circle is formed within a square • The circle has unit radius • Therefore the square has sides of length 2 • Area of the square is 4

  26. 2.3 Monte Carlo Methods

  27. 2.3 Monte Carlo Methods • The ratio of the area of the circle to the square is • (p r2)/2x2 = p/4 • Points within the square are chosen randomly and a score is kept of how many points happen to lie within the circle • The fraction of points within the circle will be p/4, given a sufficient number of randomly selected samples.

  28. 2.3 Monte Carlo Methods • Random Number Generation • The most popular way of creating a pueudorandom number sequence: • x1, x2, x3, …, xi-1, xi, xi+1, …, xn-1, xn, • is by evaluating xi+1 from a carefully chosen function of xi, often of the form • xi+1 = (axi+ c) mod m • where a, c, and m are constants chosen to create a sequence that has similar properties to truly random sequences.

  29. 2.3 Monte Carlo Methods • Parallel Random Number Generation • It turns out that • xi+1 = (axi+ c) mod m • xi+k= (Axi+ C) mod m • where • A = akmod m, • C = c(ak-1 + an-2 + … + a1 + a0) mod m, • and k is a selected “jump” constant.

  30. 5. Summary • This Chapter introduced the following concepts: • An ideal embarrassingly parallel computation • Embarrassingly parallel problems and analyses • Partitioning a two-dimensional data set • Work pool approach to achieve load balancing • Counter termination algorithm • Monte Carlo methods • Parallel random number generation

More Related