1 / 11

Parallel Performance

Week 7 Lecture Notes. Parallel Performance. Parallel Speedup. Parallel speedup: how much faster does my code run in parallel compared to serial? Max value is p , the number of processors. Parallel speedup = serial execution time

edaryl
Download Presentation

Parallel Performance

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Week 7 Lecture Notes Parallel Performance

  2. Parallel Speedup • Parallel speedup: how much faster does my code run in parallel compared to serial? • Max value is p, the number of processors Parallel speedup = serial execution time parallel execution time 2

  3. Parallel Efficiency • Parallel efficiency: how much faster does my code run in parallel compared to linear speedup from serial? • Max value is 1 Parallel efficiency = Parallel speedup p 3

  4. Model of Parallel Execution Time • Let n represent the problem size and p be the number of processors • Execution time of the inherently sequential portion of the code is s (n) • Execution time of the parallelizable portion of the code f (n) / p • Overhead is due parallel communication, synchronization is k (n,p) T(n,p) = s(n) + j (n) / p + k(n,p)

  5. Model of Parallel Speedup • Divide T(n,1) by T(n,p) • Cute notation by Quinn: y (or Greek psi) = parallel speedup s(n) + j (n) y (n,p) = s(n) + j (n) / p + k(n,p) 5

  6. Amdahl’s Law • Parallel overhead k is always positive, so… • Can express it in terms of sequential fraction f = s (n) / [s (n) + j (n)] • Divide numerator, denominator by [s (n) + j (n)] s(n) + j (n) y (n,p) < s(n) + j (n) / p 6

  7. Other Expressions for Amdahl’s Law 1 y (n,p) < f + (1 - f ) / p p y (n,p) < 1 + f (p - 1) 7

  8. Parallel Overhead: Synchronization (Example) • Suppose the parallelizable part of the code consists of n tasks each taking the same time t • What if n is not divisible by p? Let x = mod(n,p) = the “extra” tasks • Sequential execution time: t n = t (n - x) + t x • For parallel execution, only the first term shrinks inversely with p • For p processors, the second term takes either 0 or t (because x < p) • Therefore, time on p processors = t (n - x) / p + t {1 - d0,x} T(n,p) = s(n) + j (n) / p + k(n,p) j(n) = t [n – mod(n,p)], k(n,p) = t {1 - d0,mod(n,p)}

  9. Aside: How Do You Code a Kronecker Delta in C? • Simple once you see it… not so simple to come up with it… • Formula assumes 0 <= x < p Kron(0,x) = 1 - (x + (p-x)%p)/p)

  10. Parallel Overhead: Jitter (Example)

  11. Communication • Latency • Bandwidth

More Related