10/11: Lecture Topics

1. 10/11: Lecture Topics • Execution cycle • Introduction to pipelining • Data hazards

2. Office Hours • Changing Th 8:30-9:30 to Mo 2-3 • New office hours are • Mo 10:30-11:30 • Mo 2:00-3:00 • Tu 2:30-3:30

3. IF ID EX MEM WB Execution Cycle • Five steps to executing an instruction: 1. Fetch • Get the next instruction to execute from memory onto the chip 2. Decode • Figure out what the instruction says to do • Get values from registers 3. Execute • Do what the instruction says; for example, • On a memory reference, add up base and offset • On an arithmetic instruction, do the math

4. IF ID EX MEM WB More Execution Cycle 4. Memory Access • If it’s a load or store, access memory • If it’s a branch, replace the PC with the destination address • Otherwise do nothing 5. Write back • Place the result of the operation in the appropriate register

5. add $s0, $s1, $s2 • IF get instruction at PC from memory • it’s 000000 10001 10010 10000 00000 100000 • ID determine what 000000 … 100000 is • 000000 … 100000 is add • get contents of $s1 and $s2 ($s1=7, $s2=12) • EX add 7 and 12 = 19 • MEM do nothing • WB store 19 in $s0

6. lw $t2, 16($s0) • IF get instruction at PC from memory • it’s 010111 10000 01000 0000000000010000 • ID determine what 010111 is • 010111 is lw • get contents of $s0 and $t2 (we don’t know that we don’t care about $t2) $s0=0x200D1C00, $t2=77763 • EX add 16 to 0x200D1C00 = 0x200D1C10 • MEM load the word stored at 0x200D1C10 • WB store loaded value in $t2

7. IF 6 1 IF 7 ID 2 ID 3 EX EX 8 MEM 4 MEM 9 5 WB WB 10 inst 1 inst 2 Latency & Throughput • Latency—the time it takes for an individual instruction to execute • What’s the latency for this implementation? • Throughput—the number of instructions that execute per minute • What’s the throughput of this implementation?

8. A case for pipelining • The functional units are being underutilized • the instruction fetcher is used once every five clock cycles • why not have it fetch a new instruction every clock cycle? • Pipelining overlaps the stages of execution so every stage has something to due each cycle • A pipeline with N stages could speedup by N times, but • each stage must take the same amount of time • each stage must always have work to do • Also, latency for each instruction may go up, but why don’t we care?

9. Unpipelined Assembly Line • What is the latency of this assembly line, i.e. for how many cycles is the plane on the assembly line? • What is the throughput of this assembly line, i.e. how many planes are manufactured each cycle?

10. Pipelined Assembly Line • The assembly line has 5 stages • If a plane isn’t ready to go to the next stage then the pipeline stalls • that stage and all stages before it freeze • The gap in the assembly line is known as a bubble

11. Pipelined Analysis • What is the latency? • What is the throughput? • What is the speed up? (Speed up = Old Time / New Time)

12. IF ID EX MEM WB Pipeline Example

13. 1 2 3 4 5 6 7 8 9 IF IF IF IF IF ID ID ID ID ID EX EX EX EX EX MEM MEM MEM MEM MEM WB WB WB WB WB inst 1 inst 3 inst 4 inst 2 inst 5 Pipelined Xput and Latency • What’s the throughput of this implementation? • What’s the latency of this implementation?

14. IF IF ID ID EX EX MEM MEM WB WB Data Hazards • What happens in the following code? add $s0, $s1, $s2 add $s4, $s3, $s0 $s0 is read here $s0 is written here • This is called as a data dependency • When it causes a pipeline stall it is called a data hazard

15. IF ID EX MEM WB ID EX MEM WB Solution: Stall • Stall the pipeline until the result is available add s0,s1,s2 IF add s4,s3,s0 stall • Stall the pipeline until the result is available

16. IF ID EX MEM WB write $s0 ID EX MEM WB add s0,s1,s2 read $s0 IF add s4,s3,s0 stall Solution: Read & Write in same Cycle • Write the register in the first part of the clock cycle • Read it in the second part of the clock cycle • A stall of two cycles is still required

17. IF IF ID ID EX EX MEM MEM WB WB add s0,s1,s2 add s4,s3,s0 Solution: Forwarding • The value of $s0 is known after cycle 3 (after the first instruction’s EX stage) • The value of $s0 isn’t needed until cycle 4 (before the second instruction’s EX stage) • If we forward the result there isn’t a stall

18. IF IF ID ID EX EX MEM MEM WB WB lw s0,0(s2) add s4,s3,s0 Another data hazard • What if the first instruction is lw? • s0 isn’t known until after the MEM stage • We can’t forward back into the past • Either stall or reorder instructions

19. IF IF IF IF ID ID ID ID EX EX EX EX MEM MEM MEM MEM WB WB WB WB lw s0,0(s2) add s4,s3,s0 IF ID EX MEM WB Solutions to the lw hazard • We can stall for one cycle, but we hate to stall • Try to execute an unrelated instruction between the two instructions lw s0,0(s2) sub t4,t2,t3 add s4,s3,s0 sub t4,t2,t3

20. Reordering Instructions • Reordering instructions is a common technique for avoiding pipeline stalls • Sometimes the compiler does the reordering statically • Almost all modern processors do this reordering dynamically • they can see several instructions and they execute anyone that has no dependency • this is known as out-of-order execution and is very complicated to implement

21. IF IF ID ID EX EX MEM MEM WB WB Control Hazards • Branch instructions cause control hazards because we don’t know which instruction to execute next bne $s0, $s1, next add $s4, $s3, $s0 ... next: sub $s4, $s3, $s0 do we fetch add or sub? we don’t know until here