1 / 24

Lecture 9 Software Pipelining

Lecture 9 Software Pipelining. Introduction Problem Formulation Algorithm. I. Example of DoAll Loops. Machine: Per clock: 1 read , 1 write , 1 (2-stage) arithmetic op , with hardware loop op and auto-incrementing addressing mode. Source code:

levi
Download Presentation

Lecture 9 Software Pipelining

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 9Software Pipelining Introduction Problem Formulation Algorithm With slides “borrowed” from M. Lam

  2. I. Example of DoAll Loops CS243: Software Pipelining • Machine: • Per clock: 1 read, 1 write, 1 (2-stage) arithmetic op, with hardware loop op and auto-incrementing addressing mode. • Source code: For i = 1 to n D[i] = A[i] * B[i]+ c • Code for one iteration: 1. LD R5,0(R1++) 2. LD R6,0(R2++) 3. MUL R7,R5,R6 4. 5. ADD R8,R7,R4 6. 7. ST 0(R3++),R8 • No parallelism in basic block

  3. Unrolling CS243: Software Pipelining 1.L: LD 2. LD 3. LD 4. MUL LD 5. MUL LD 6. ADD LD 7. ADD LD 8. ST MUL LD 9. MUL 10. ST ADD • ADD • ST 13. ST BL (L) • Let u be the degree of unrolling: • Length of u iterations = 7+2(u-1) • Execution time per source iteration = (7+2(u-1)) / u = 2 + 5/u

  4. Software Pipelined Code CS243: Software Pipelining 1. LD 2. LD 3. MUL LD 4. LD 5. MUL LD 6. ADD LD 7. MUL LD 8. ST ADD LD 9. MUL LD 10. ST ADD LD MUL ST ADD 13. ST ADD 15. 16. ST Unlike unrolling, software pipelining can give optimal result. Locally compacted code may not be globally optimal DOALL: Can fill arbitrarily long pipelines with infinitely many iterations

  5. Example of DoAcross Loop LD MUL ADD ST CS243: Software Pipelining Loop: Sum = Sum + A[i]; B[i] = A[i] * c; Software Pipelined Code 1. LD 2. MUL3. ADD LD4. ST MUL 5. ADD 6. ST Doacross loops Recurrences can be parallelized Harder to fully utilize hardware with large degrees of parallelism

  6. II. Problem Formulation S 0 LD 1 MUL2 ADD LD3 ST MUL ADD ST T=2 CS243: Software Pipelining Goals: • maximize throughput • small code size Find: • an identicalrelative schedule S(n) for every iteration • a constantinitiation interval (T) such that • the initiation interval is minimized Complexity: • NP-complete in general

  7. Resources on Bound on Initiation Interval CS243: Software Pipelining • Example: Resource usage of 1 iteration; Machine can execute 1 LD, 1 ST, 2 ALU per clock LD, LD, MUL, ADD, ST • Lower bound on initiation interval? for all resource i, number of units required by one iteration: ni number of units in system: Ri Lower bound due to resource constraints: maxini/Ri

  8. Scheduling Constraints: Resource Iteration 1 LD Alu ST Iteration 2 T=2 LD Alu ST Iteration 3 LD Alu ST Iteration 4 Steady State Time LD Alu ST LD Alu ST T=2 CS243: Software Pipelining RT: resource reservation table for single iteration RTs: modulo resource reservation table RTs[i] = t|(t mod T = i) RT[t]

  9. Scheduling Constraints: Precedence CS243: Software Pipelining for (i = 0; i < n; i++) { *(p++) = *(q++) + c } • Minimum initiation interval? • S(n): Schedule for n with respect to the beginning of the schedule • Label edges with < , d > •  = iteration difference, d = delay  x T + S(n2) – S(n1)  d

  10. Scheduling Constraints: Precedence CS243: Software Pipelining for (i = 2; i < n; i++) { A[i] = A[i-2] + 1; } • Minimum initiation interval? • S(n): Schedule for n with respect to the beginning of the schedule • Label edges with < , d > •  = iteration difference, d = delay  x T + S(n2) – S(n1)  d

  11. Minimum Initiation Interval CS243: Software Pipelining For all cycles c, max cCycleLength(c) / IterationDifference (c)

  12. III. Example: An Acyclic Graph CS243: Software Pipelining

  13. Algorithm for Acyclic Graphs CS243: Software Pipelining • Find lower bound of initiation interval: T0 • based on resource constraints • For T = T0, T0+1, ... until all nodes are scheduled • For each node n in topological order • s0 = earliest n can be scheduled • for each s = s0 , s0 +1, ..., s0 +T-1 • if NodeScheduled(n, s) break; • if n cannot be scheduled break; • NodeScheduled(n, s) • Check resources of n at s in modulo resource reservation table • Tractable if: • every operation uses 1 resource, and • no cyclic dependences in the loop

  14. Cyclic Graphs CS243: Software Pipelining No such thing as “topological order” b  c; c  b S(c) – S(b)  1 T + S(b) – S(c)  2 Scheduling b constrains c and vice versa S(b) + 1  S(c)  S(b) – 2 + T S(c) – T + 2  S(b)  S(c) – 1

  15. Strongly Connected Components CS243: Software Pipelining • Astrongly connected component(SCC) • Set of nodes such that every node can reach every other node • Every node constrains all others from above and below • Finds longest paths between every pair of nodes • As each node scheduled, find lower and upper bounds of all other nodes in SCC • SCCs are hard to schedule • Critical cycle: no slack • Backtrack starting with the first node in SCC • increases T, increases slack • Edges between SCCs are acyclic • Acyclic graph: every node is a separate SCC

  16. Algorithm Design CS243: Software Pipelining • Find lower bound of initiation interval: T0 • based on resource constraints and precedence constraints • For T = T0, T0+1, ... , until all nodes are scheduled • E*= longest path between each pair • For each SCC c in topological order • s0 = Earliest c can be scheduled • For each s = s0 , s0 +1, ..., s0 +T-1 • if SCCScheduled(c, s) break; • If c cannot be scheduled return false; • return true;

  17. Scheduling a Strongly Connected Component (SCC) CS243: Software Pipelining • SCCScheduled(c, s) • Schedule first node at s, return false if fails • For each remaining node n in c • sl = lower bound on n based on E* • su = upper bound on n based on E* • For each s = sl , sl +1, min (sl +T-1, su) • if NodeScheduled(n, s) break; • If n cannot be scheduled return false; • return true;

  18. Modulo Variable Expansion 1. LD R5,0(R1++) 2. LD R6,0(R2++) 3. MUL R7,R5,R6 4. 5. ADD R8,R7,R4 6. 7. ST 0(R3++),R8 CS243: Software Pipelining Software-pipelined code 1. LD 2. LD 3. MUL LD 4. LD 5. MUL LD 6. ADD LD L:7. MUL LD 8. ST ADD LD BL L 9. MUL LD 10. ST ADD LD 11. MUL 12. ST ADD 13. 14. ST ADD

  19. Modulo Variable Expansion CS243: Software Pipelining 1. LD R5,0(R1++) 2. LD R6,0(R1++) 3. LD R5,0(R1++) MUL R7,R5,R6 4. LD R6,0(R1++) 5. LD R5,0(R1++) MUL R17,R5,R6 6. LD R6,0(R1++) ADD R8,R7,R7 L 7. LD R5,0(R1++) MUL R7,R5,R6 8. LD R6,0(R1++) ADD R8,R17,R17 ST 0(R3++),R8 9. LD R5,0(R1++) MUL R17,R5,R6 10. LD R6,0(R1++) ADD R8,R7,R7 ST 0(R3++),R8 BL L 11. MUL R7,R5,R6 12. ADD R8,R17,R17 ST 0(R3++),R8 13. 14. ADD R8,R7,R7 ST 0(R3++),R8 15. 16. ST 0(R3++),R8

  20. Algorithm CS243: Software Pipelining • Normally, every iteration uses the same set of registers • introduces artificial anti-dependences for software pipelining • Modulo variable expansion algorithm • schedule each iteration ignoring artificial constraints on registers • calculate life times of registers • degree of unrolling = maxr (lifetimer /T) • unroll the steady state of software pipelined loop to use different registers • Code generation • generate one pipelined loop with only one exit (at beginning of steady state) • generate one unpipelined loop to handle the rest • code generation is the messiest part of the algorithm!

  21. Conclusions CS243: Software Pipelining • Numerical Code • Software pipelining is useful for machines with a lot of pipelining and instruction level parallelism • Compact code • Limits to parallelism: dependences, critical resource

  22. Integer Linear Programming CS243: Software Pipelining • Gao et al. • Shows software pipelining + resource constraints + register alloccan be solved as an integer programming problem. • Comparison between heuristics and optimal methods • Software Pipelining Showdown: Optimal vs. Heuristic Methods in a Production Compiler [Ruttenberg, Gao, Stoutchinin, Lichtenstein, PLDI, 1996] • Software pipelining heuristics with backtracking tuned for MIPS R8000 (4-issue statically scheduled) • Improves SPECfp92 from 203.03 to 325.27 (some programs improve by 2.5 times) • Result: • Optimal solution rarely does better • Optimal takes much longer, and sometimes fails to finish (esp. with register opt). • Heuristics can be better than “optimal” because ILP problem does not model all possible optimizations (e.g. memory bank optimization)

  23. Integer Linear Programming Formulation CS243: Software Pipelining Let P be target initiation interval Let Ti be time of execution of node from the start of iteration Ti = Ki P + xi Represent schedule in integer linear programming form, exposing the modulo resource utilization Execute once: aij = 0 or 1, Sum of row = 1 Resource reservation, I(r): set of instruction using r, with Rr units Precedence constraint: Let mij be iteration difference, di be delay Tj - Tidi – P mij

  24. Objective Function CS243: Software Pipelining • Any schedule that satisfies all the constraints is sufficient. • To help guide the ILP, use objective function: • Minimize: where r is a resource type, Rr is the number of resources of type r,Cr is a cost introduced for example to measure the criticality of resource r.

More Related