ECE200 – Computer Organization

1. ECE200 – Computer Organization Chapter 6 – Enhancing Performance with Pipelining

2. Homework 6 • 6.2, 6.3, 6.5, 6.9, 6.11, 6.19, 6.27, 6.30

3. Outline for Chapter 6 lectures • Pipeline motivation: increasing instruction throughput • MIPS 5-stage pipeline • Hazards • Handling exceptions • Superscalar execution • Dynamic scheduling (out-of-order execution) • Real pipeline designs

4. Pipeline motivation • Need both low CPI and high frequency for best performance • Want a multicycle for high frequency, but need better CPI • Idea behind pipelining is to have a multicycle implementation that operates like a factory assembly line • Each “worker” in the pipeline performs a particular task, hands off to the next “worker”, while getting new work

5. Pipeline motivation • Tasks should take about the same time – if one “worker” is much slower than the rest, then other “workers” will stand idle • Once the assembly line is full, a new “product” (instruction) comes out of the back-end of the line each time period • In a computer assembly line (pipeline), each task is called a stage and the time period is one clock cycle

6. MIPS 5-stage pipeline • Like single cycle datapath but with registers separating each stage

7. MIPS 5-stage pipeline • 5 stages for each instruction • IF: instruction fetch • ID: instruction decode and register file read • EX: instruction execution or effective address calculation • MEM: memory access for load and store • WB: write back results to register file • Delays of all 5 stages are relatively the same • Staging registers are used to hold data and control as instructions pass between stages • All instructions pass through all 5 stages • As an instruction leaves a stage in a particular clock period, the next instruction enters it

8. Pipeline operation for lw • Stage 1: Instruction fetch

9. Pipeline operation for lw • Stage 2: Instruction decode and register file read • What happens to the instruction info in IF/ID?

10. Pipeline operation for lw • Stage 3: Effective address calculation

11. Pipeline operation for lw • Stage 4: Memory access

12. Pipeline operation for lw • Stage 5: Write back • Instruction info in IF/ID is gone – won’t work

13. Modified pipeline with write back fix • Write register bits from the instruction must be carried through the pipeline with the instruction

14. Pipeline operation for lw • Pipeline usage in each stage for lw

15. Pipeline operation for sw • Stage 3: Effective address calculation

16. Pipeline operation for sw • Stage 4: Memory access

17. Pipeline operation for sw • Stage 5: Write back (nothing)

18. Pipeline operation for lw, sub sequence

19. Pipeline operation for lw, sub sequence

20. Pipeline operation for lw, sub sequence

21. Pipeline operation for lw, sub sequence

22. Pipeline operation for lw, sub sequence

23. Pipeline operation for lw, sub sequence

24. Graphical pipeline representation • Represent overlap of pipelined instructions as multiple pipelines skewed by a cycle

25. Another useful shorthand form

26. Pipeline control • Basic pipeline control is similar to the single cycle implementation

27. Pipeline control • Control for an instruction is generated in ID and travels with the instruction and data through the pipeline • When an instruction enters a stage, it’s control signals set the operation of that stage

28. Pipeline control

29. Multiple instruction example • For the following code fragment show the datapath and control usage as the instruction sequence travels down the pipeline lw $10, 20($1) sub $11, $2, $3 and $12, $4, $5 or $13, $6, $7 add $14, $8, $9

30. Multiple instruction example

31. Multiple instruction example

32. Multiple instruction example

33. Multiple instruction example

34. Multiple instruction example

35. Multiple instruction example

36. Multiple instruction example

37. Multiple instruction example

38. Multiple instruction example

39. How the MIPS ISA simplifies pipelining • Fixed length instruction simplifies • Fetch – just get the next 32 bits • Decode – single step; don’t have to decode opcode before figuring out where to get the rest of the fields • Source register fields always in same location • Can read source registers during decode • Load/store architecture • ALU can be used for both arithmetic and EA calculation • Memory instruction require about same amount of work as arithmetic ones, easing pipelining of the two together • Memory data must be aligned • Read or write accesses can be done in one cycle

40. Pipeline hazards • A hazard is a conflict, regarding data, control, or hardware resources • Data hazards are conflicts for register values • Control hazards occur due to the delay to execute branch and jump instruction • Structural hazards are conflicts for hardware resources, such as • A single memory for instructions and data • A multi-cycle, non-pipelined functional unit (such as a divider)

41. Data dependences • A read after write (RAW) dependence occurs when the register written by an instruction is a source register of a subsequent instruction • Also have write after read (WAR) and write after write (WAW) data dependences (later) lw $10, 20($1) sub $11, $10, $3 and $12, $4, $11 or $13, $11, $4 add $14, $13, $9

42. Pipelining and RAW dependences • RAW dependences that are close by may cause data hazards in the pipeline • Consider the following code sequence: • What are the RAW dependences? sub $2, $1, $3 and $12, $2, $6 or $13, $6, $2 add $14, $2, $2 sw $15, 100($2)

43. Pipelining and RAW dependences • Data hazards with first three instructions hazard hazard ok ok

44. Forwarding • Most RAW hazards can be eliminated by forwarding results between pipe stages at this point, result of sub is available

45. Forwarding datapaths • Bypass paths feed data from MEM and WB back to MUXes at the EX ALU inputs • Do we still have to write the register file in WB?

46. Detecting forwarding • Rd of the instruction in MEM or WB must match Rs and/or Rt of the instruction in EX • The instruction in MEM or WB must have RegWrite=1 (why?) • Rd must not be $0 (why?)

47. Detecting forwarding from MEM to EX • To the upper ALU input (ALUupper) • EX/MEM.RegWrite = 1 • EX/MEM.RegisterRd not equal 0 • EX/MEM.RegisterRd = ID/EX.RegisterRs • To the lower ALU input (ALUlower) • EX/MEM.RegWrite = 1 • EX/MEM.RegisterRd not equal 0 • EX/MEM.RegisterRd = ID/EX.RegisterRt

48. Detecting forwarding from WB to EX • To the upper ALU input • MEM/WB.RegWrite = 1 • MEM/WB.RegisterRd not equal 0 • MEM/WB.RegisterRd = ID/EX.RegisterRs • The value is not being forwarded from MEM (why?) • To the lower ALU input • MEM/WB.RegWrite = 1 • MEM/WB.RegisterRd not equal 0 • MEM/WB.RegisterRd = ID/EX.RegisterRt • The value is not being forwarded from MEM

49. Forwarding control • Control is handled by the forwarding unit

50. Forwarding example • Show forwarding for the code sequence: sub $2, $1, $3 and $4, $2, $5 or $4, $4, $2 add $9, $4, $2