Automatic Array Privatization for Parallel Computing

1. Automatic Array Privatization Peng Tu and David Padua In Proceedings of the 6th Annual Workshop on Languages and Compilers for Parallel Computing, August 1993. Presented by: Karl Cheng

2. Contribution • data flow-based vs. dependence analysis • Extensible for interprocedural analysis • Distinguish static and dynamic last value assignment

3. Data Flow Framework • For the loop below: • X(1:N) = 0 • DO I = 1,N • A(1) = X(I) • DO J=2,N • Y(J) = J • A(J) = A(J-1)+Y(J) • ENDDO • ENDDO • The flow graph looks like: X(I) J A(1) Y(J) A(J)

4. Data Flow framework • For basic block S and the set of variables VAR in the program, we define: • DEF(S) = {v in VAR: v has outward exposed definition in S} • USE(S) = {v in VAR: v has outward exposed use in S} • KILL(S) = {v in VAR: v has a definition in S}

5. Data Flow framework • For the basic block in the inner loop of the previous program: • Y(J) = J • A(J) = A(J-1)+Y(J) • We define: • DEF(S) = {A(J), Y(J)} • USE(S) = {A(J-1),J} • KILL(S) = {A(J),Y(J)} J A(J-1) Y(J) A(J)

6. MRD • MRDin(S): set of variables always defined upon entering S • MRDin(S) = ∩t in pred(S) MRDout(t) • MRDout(S): set of variables always defined upon exiting S • MRDout(S) = (MRDin(S) – KILL(S)) U DEF(S)

7. MRD • X(1:N) = 0 • DO I = 1,N • A(1) = X(I) • DO J=2,N • Y(J) = J • A(J) = A(J-1)+Y(J) • ENDDO • ENDDO • Remember: • DEF(S) = {A(J), Y(J)} • USE(S) = {A(J-1),J} • KILL(S) = {A(J),Y(J)} • Now: • MRDin (S) = ∩t in pred(S) MRDout(t) = {J} • MRDout (S) = (MRDin(S) – KILL(S)) U DEF(S) = {A(J), Y(J), J} • Back edges are removed X(I) J A(J-1) Y(J) A(J) S

8. Abstraction of inner loops • Extends definition for basic block to 1iteration of loop body • DEFb(L) = must defined variables for 1 iteration of Loop L • DEFb(L) = ∩ (MRDout(t) : t in exits (L) • USEb(L) = possibly outward exposed variables • USEb(L) = U (USE(t) – MRDin(t)) : t in body (L) • KILLb(L) = DEFb(L) • PRIb(L) = privatizable variables: used but not exposed to definition outside of iteration • PRIb(L) = U { USE(t) : t in body(L)} - USEb(L)

9. Abstraction of inner loops • X(1:N) = 0 • DO I = 1,N • A(1) = X(I) • DO J=2,N • Y(J) = J • A(J) = A(J-1)+Y(J) • ENDDO • ENDDO • Remember: • MRDin (S) = ∩t in pred(S) MRDout(t) = {J} • MRDout (S) = (MRDin(S) – KILL(S)) U DEF(S) = {A(J), Y(J), J} • Now: • DEFb(L) = ∩ (MRDout(t) : t in exits (L) = {A(J), Y(J), J} • USEb(L) = U (USE(t) – MRDin(t)) : t in body (L) = {A(J-1)} • PRIb(L) = U { USE(t) : t in body(L)} - USEb(L) = {J, Y(J)} X(I) J A(J-1) Y(J) A(J) S

10. Abstraction of inner loops • For analysis of outer loops, we need the total effects of inner loops • List summary sets for each iteration • Use aggregated set (approximation) • If index I = [1:N], aggregate(A(I,J)) = A(1:N,J) • After this, we can treat inner loop as single node in the flow graph

11. Privatization Algorithm • For each statement t in body (L), if t is a nested loop, recurse to get [DEF(t),USE(t)]. Otherwise compute local DEF(t), USE(t) • Scan the loop again, contruct MRDin and MRDout • Compute Summary set for loop body • Aggregate to get the privatizable set of the entire loop • Return [DEF, USE]

12. Privatization Algorithm • X(1:N) = 0 • DO I = 1,N • A(1) = X(I) • DO J=2,N • Y(J) = J • A(J) = A(J-1)+Y(J) • ENDDO • ENDDO • DEFb(L) = ∩ (MRDout(t) : t in exits (L) = {A(1:N), Y(2:N),J,I} • USEb(L) = U (USE(t) – MRDin(t)) : t in body (L) = {X(I)} • PRIb(L) = U { USE(t) : t in body(L)} - USEb(L) = {J, I, A(1:N), Y(2:N)} X(I) A(1) DEF = {A(2:N), Y(2:N)} USE = {A(1)}

13. Profitability Test • Consider this loop: • DO I = 1,N • A(I) = … • … = A (I) • Profitability Test: If different iterations access different A(i)  not profitable 

14. Last Value Assignment • Live Analysis • LVBOT(S) = U t in succ(S) LVTOP(t) • LVTOP(S) = (LVBOT(S) – KILL(S)) U USE(S) • Static vs Dynamic • PRIbst = PRIb∩DEFb (DEFb(L) = must defined variables for 1 iteration of Loop L)

15. Automatic vs Manual Array Privatization

16. Thank You Questions?