L ECTURE 2. Recurrences (Review). Matrix Multiplication. Merge Sort. Tableau Construction. Conclusion. The Master Method. The Master Method for solving recurrences applies to recurrences of the form T ( n ) = a T ( n / b ) + f ( n ) ,
An Image/Link below is provided (as is) to download presentation
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.Content is provided to you AS IS for your information and personal use only.Download presentation by click this link.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.During download, if you can't get a presentation, the file might be deleted by the publisher.
The Master Method The Master Method for solving recurrences applies to recurrences of the form T(n) = aT(n/b) + f(n), where a¸ 1, b > 1, and f is asymptotically positive. * IDEA: Compare nlogba with f(n). * The unstated base case is T(n) = (1) for sufficiently small n.
Master Method — CASE 1 T(n) = aT(n/b) + f(n) nlogbaÀ f (n) Specifically, f(n) = O(nlogba – e) for some constant e > 0. Solution:T(n) = Q(nlogba) .
Master Method — CASE 2 T(n) = aT(n/b) + f(n) nlogba¼ f (n) Specifically, f(n) = (nlogbalgkn) for some constant k¸ 0. Solution:T(n) = Q(nlogba lgk+1n) .
Master Method — CASE 3 T(n) = aT(n/b) + f(n) nlogba¿ f (n) Specifically, f(n) = (nlogba + e) for some constant e > 0 andf(n)satisfies the regularity conditionthat af(n/b) ·cf(n)for some constant c < 1. Solution:T(n) = Q(f(n)) .
Square-Matrix Multiplication bn1 b21 b11 an1 a21 a11 c11 c21 cn1 an2 a22 a12 b22 bn2 b12 cn2 c12 c22 ann a2n b1n b2n bnn a1n cnn c2n c1n L L L L L L L L L M M M M M M M M M O O O n cij = aik bkj k= 1 = £ C A B Assume for simplicity that n = 2k.
Matrix Multiply in Pseudo-Cilk cilk void Mult(*C, *A, *B, n) { float *T = Cilk_alloca(n*n*sizeof(float)); hbase case & partition matrices i spawn Mult(C11,A11,B11,n/2); spawn Mult(C12,A11,B12,n/2); spawn Mult(C22,A21,B12,n/2); spawn Mult(C21,A21,B11,n/2); spawn Mult(T11,A12,B21,n/2); spawn Mult(T12,A12,B22,n/2); spawn Mult(T22,A22,B22,n/2); spawn Mult(T21,A22,B21,n/2); sync; spawn Add(C,T,n); sync; return; } Absence of type declarations. C = A¢ B
Matrix Multiply in Pseudo-Cilk cilk void Mult(*C, *A, *B, n) { float *T = Cilk_alloca(n*n*sizeof(float)); hbase case & partition matrices i spawn Mult(C11,A11,B11,n/2); spawn Mult(C12,A11,B12,n/2); spawn Mult(C22,A21,B12,n/2); spawn Mult(C21,A21,B11,n/2); spawn Mult(T11,A12,B21,n/2); spawn Mult(T12,A12,B22,n/2); spawn Mult(T22,A22,B22,n/2); spawn Mult(T21,A22,B21,n/2); sync; spawn Add(C,T,n); sync; return; } Coarsen base cases for efficiency. C = A¢ B
Matrix Multiply in Pseudo-Cilk cilk void Mult(*C, *A, *B, n) { float *T = Cilk_alloca(n*n*sizeof(float)); hbase case & partition matrices i spawn Mult(C11,A11,B11,n/2); spawn Mult(C12,A11,B12,n/2); spawn Mult(C22,A21,B12,n/2); spawn Mult(C21,A21,B11,n/2); spawn Mult(T11,A12,B21,n/2); spawn Mult(T12,A12,B22,n/2); spawn Mult(T22,A22,B22,n/2); spawn Mult(T21,A22,B21,n/2); sync; spawn Add(C,T,n); sync; return; } Also need a row-size argument for array indexing. Submatrices are produced by pointer calculation, not copying of elements. C = A¢ B
Matrix Multiply in Pseudo-Cilk cilk void Mult(*C, *A, *B, n) { float *T = Cilk_alloca(n*n*sizeof(float)); hbase case & partition matrices i spawn Mult(C11,A11,B11,n/2); spawn Mult(C12,A11,B12,n/2); spawn Mult(C22,A21,B12,n/2); spawn Mult(C21,A21,B11,n/2); spawn Mult(T11,A12,B21,n/2); spawn Mult(T12,A12,B22,n/2); spawn Mult(T22,A22,B22,n/2); spawn Mult(T21,A22,B21,n/2); sync; spawn Add(C,T,n); sync; return; } cilkvoid Add(*C, *T, n) { h base case & partition matrices i spawn Add(C11,T11,n/2); spawn Add(C12,T12,n/2); spawn Add(C21,T21,n/2); spawn Add(C22,T22,n/2); sync; return; } C = A¢ B C = C + T
Work of Matrix Addition cilk void Add(*C, *T, n) { hbase case & partition matrices i spawn Add(C11,T11,n/2); spawn Add(C12,T12,n/2); spawn Add(C21,T21,n/2); spawn Add(C22,T22,n/2); sync; return; } Work: A1(n) = ? 4A1(n/2) + (1) — CASE 1 = (n2) nlogba = nlog24 = n2À(1).
Stack Temporaries cilk void Mult(*C, *A, *B, n) { hbase case & partition matrices i spawn Mult(C11,A11,B11,n/2); spawn Mult(C12,A11,B12,n/2); spawn Mult(T21,A22,B21,n/2); sync; spawn Add(C,T,n); sync; return; } float *T = Cilk_alloca(n*n*sizeof(float)); In hierarchical-memory machines (especially chip multiprocessors), memory accesses are so expensive that minimizing storage often yields higher performance. IDEA: Trade off parallelism for less storage.
No-Temp Matrix Multiplication cilk void MultA(*C, *A, *B, n) { // C = C + A * B hbase case & partition matrices i spawn MultA(C11,A11,B11,n/2); spawn MultA(C12,A11,B12,n/2); spawn MultA(C22,A21,B12,n/2); spawn MultA(C21,A21,B11,n/2); sync; spawn MultA(C21,A22,B21,n/2); spawn MultA(C22,A22,B22,n/2); spawnMultA(C12,A12,B22,n/2); spawnMultA(C11,A12,B21,n/2); sync; return; } Saves space, but at what expense?
Work of No-Temp Multiply cilk void MultA(*C, *A, *B, n) { // C = C + A * B hbase case & partition matrices i spawn MultA(C11,A11,B11,n/2); spawn MultA(C12,A11,B12,n/2); spawn MultA(C22,A21,B12,n/2); spawn MultA(C21,A21,B11,n/2); sync; spawn MultA(C21,A22,B21,n/2); spawn MultA(C22,A22,B22,n/2); spawnMultA(C12,A12,B22,n/2); spawnMultA(C11,A12,B21,n/2); sync; return; } Work: M1(n) = ? 8 M1(n/2) + (1) — CASE 1 =(n3)
Span of No-Temp Multiply cilk void MultA(*C, *A, *B, n) { // C = C + A * B hbase case & partition matrices i spawn MultA(C11,A11,B11,n/2); spawn MultA(C12,A11,B12,n/2); spawn MultA(C22,A21,B12,n/2); spawn MultA(C21,A21,B11,n/2); sync; spawn MultA(C21,A22,B21,n/2); spawn MultA(C22,A22,B22,n/2); spawnMultA(C12,A12,B22,n/2); spawnMultA(C11,A12,B21,n/2); sync; return; } cilk void MultA(*C, *A, *B, n) { // C = C + A * B hbase case & partition matrices i spawn MultA(C11,A11,B11,n/2); spawn MultA(C12,A11,B12,n/2); spawn MultA(C22,A21,B12,n/2); spawn MultA(C21,A21,B11,n/2); sync; spawn MultA(C21,A22,B21,n/2); spawn MultA(C22,A22,B22,n/2); spawn MultA(C12,A12,B22,n/2); spawnMultA(C11,A12,B21,n/2); sync; return; } maximum maximum Span: M1(n) = ? 2 M1(n/2) + (1) — CASE 1 =(n)
Parallelism of No-Temp Multiply Work: M1(n) = (n3) Span: M1(n) = (n) Parallelism: M1(n) = (n2) M1(n) For 1000£1000 matrices, parallelism ¼ (103)3/103 = 106. Faster in practice!
Testing Synchronization • Cilk language feature: A programmer can check whether a Cilk procedure is “synched” (without actually performing a sync) by testing the pseudovariableSYNCHED: • SYNCHED= 0) some spawned children might not have returned. • SYNCHED= 1) all spawned children have definitely returned.
Best of Both Worlds cilk void Mult1(*C, *A, *B, n) {// multiply & store hbase case & partition matrices i spawn Mult1(C11,A11,B11,n/2); // multiply & store spawn Mult1(C12,A11,B12,n/2); spawn Mult1(C22,A21,B12,n/2); spawn Mult1(C21,A21,B11,n/2); if (SYNCHED) { spawn MultA1(C11,A12,B21,n/2); // multiply & add spawn MultA1(C12,A12,B22,n/2); spawn MultA1(C22,A22,B22,n/2); spawn MultA1(C21,A22,B21,n/2); } else { float *T = Cilk_alloca(n*n*sizeof(float)); spawn Mult1(T11,A12,B21,n/2); // multiply & store spawn Mult1(T12,A12,B22,n/2); spawn Mult1(T22,A22,B22,n/2); spawn Mult1(T21,A22,B21,n/2); sync; spawn Add(C,T,n); // C = C + T } sync; return; } This code is just as parallel as the original, but it only uses more space if runtime parallelism actually exists.
Ordinary Matrix Multiplication IDEA: Spawn n2inner products in parallel. Compute each inner product in parallel. Work: (n3) Span: (lgn) Parallelism: (n3/lgn) n k = 1 cij = aik bkj BUT, this algorithm exhibits poor locality and does not exploit the cache hierarchy of modern microprocessors, especially CMP’s.
Work of Merge Sort cilk void MergeSort(int *B, int *A, int n) { if (n==1) { B[0] = A[0]; } else { int *C; C = (int*) Cilk_alloca(n*sizeof(int)); spawn MergeSort(C, A, n/2); spawn MergeSort(C+n/2, A+n/2, n-n/2); sync; Merge(B, C, C+n/2, n/2, n-n/2); } } Work: T1(n) = ? 2T1(n/2) + (n) — CASE 2 = (n lg n) nlogba = nlog22 = n)f(n) = (nlogba lg0n) .
Span of Merge Sort cilk void MergeSort(int *B, int *A, int n) { if (n==1) { B[0] = A[0]; } else { int *C; C = (int*) Cilk_alloca(n*sizeof(int)); spawn MergeSort(C, A, n/2); spawn MergeSort(C+n/2, A+n/2, n-n/2); sync; Merge(B, C, C+n/2, n/2, n-n/2); } } Span: T1(n) = ? T1(n/2) + (n) — CASE 3 = (n) nlogba = nlog21 = 1¿ (n) .
Parallelism of Merge Sort Work: T1(n) = (n lg n) PUNY! Span: T1(n) = (n) Parallelism: T1(n) = (lg n) T1(n) We need to parallelize the merge!
Parallel Merge na/2 ·A[na/2] ¸A[na/2] Recursive merge Recursive merge Binary search ·A[na/2] ¸A[na/2] j j+1 0 na A na¸nb B 0 nb KEY IDEA:If the total number of elements to be merged in the two arrays is n = na + nb, the total number of elements in the larger of the two recursive merges is at most ? (3/4)n .
Parallel Merge cilk void P_Merge(int *C, int *A, int *B, int na, int nb) { if (na < nb) { spawn P_Merge(C, B, A, nb, na); } else if (na==1) { if (nb == 0) { C[0] = A[0]; } else { C[0] = (A[0]<B[0]) ? A[0] : B[0]; /* minimum */ C[1] = (A[0]<B[0]) ? B[0] : A[0]; /* maximum */ } } else { int ma = na/2; int mb = BinarySearch(A[ma], B, nb); spawn P_Merge(C, A, B, ma, mb); spawn P_Merge(C+ma+mb, A+ma, B+mb, na-ma, nb-mb); sync; } } Coarsen base cases for efficiency.
Span of P_Merge cilk void P_Merge(int *C, int *A, int *B, int na, int nb) { if (na < nb) { } else { int ma = na/2; int mb = BinarySearch(A[ma], B, nb); spawn P_Merge(C, A, B, ma, mb); spawn P_Merge(C+ma+mb, A+ma, B+mb, na-ma, nb-mb); sync; } } Span: T1(n) = ? T1(3n/4) + (lg n) — CASE 2 = (lg2n) nlogba = nlog4/31 = 1)f(n) = (nlogba lg1n) .
Work of P_Merge HAIRY! cilk void P_Merge(int *C, int *A, int *B, int na, int nb) { if (na < nb) { } else { int ma = na/2; int mb = BinarySearch(A[ma], B, nb); spawn P_Merge(C, A, B, ma, mb); spawn P_Merge(C+ma+mb, A+ma, B+mb, na-ma, nb-mb); sync; } } T1(n) + T1((1–)n) + (lg n),where 1/4 ·· 3/4 . T1(n) = ? Work: CLAIM:T1(n) = (n) .
Analysis of Work Recurrence T1(n) = T1(n) + T1((1–)n) + (lg n), where 1/4 ·· 3/4 . Substitution method: Inductive hypothesis is T1(k) ·c1k – c2lg k, wherec1,c2 > 0. Prove that the relation holds, and solve for c1 and c2. T1(n) = T1(n) + T1((1–)n) + (lg n) · c1(n) – c2lg(n)+ c1((1–)n) – c2lg((1–)n)+ (lg n)
Recursive Construction n Cilk code I II spawn I; sync; spawn II; spawn III; sync; spawn IV; sync; n III IV
Recursive Construction n Cilk code I II spawn I; sync; spawn II; spawn III; sync; spawn IV; sync; n III IV 4T1(n/2) + (1) Work: T1(n) = ? = (n2) — CASE 1
Recursive Construction n Cilk code I II spawn I; sync; spawn II; spawn III; sync; spawn IV; sync; n III IV Span: T1(n) = ? 3T1(n/2) + (1) = (nlg3) — CASE 1