Instruction Times (Critical Paths)

2. Instruction Times (Critical Paths) • What is the clock cycle time assuming negligible delays for muxes, control unit, sign extend, PC access, shift left 2, wires, setup and hold times except: • Instruction and Data Memory (200 ps) • ALU and adders (200 ps) • Register File access (reads or writes) (100 ps)

3. Simplified MIPS Pipelined Datapath

4. Instruction Critical Paths • What is the clock cycle time assuming negligible delays for muxes, control unit, sign extend, PC access, shift left 2, wires, setup and hold times except: • Instruction and Data Memory (200 ps) • ALU and adders (200 ps) • Register File access (reads or writes) (100 ps)

5. Stages • Five stages, one step per stage • IF: Instruction fetch from memory • ID: Instruction decode & register read • EX: Execute operation or calculate address • MEM: Access memory operand • WB: Write result back to register

6. Hazards • Situations that prevent starting the next instruction in the next cycle • Structure hazards • Data hazard • Control hazard

7. Structure Hazards • Instruction cannot execute in proper clock cycle because hardware cannot support the combination of instructions that are set to execute in the clock cycle • In MIPS pipeline with a single memory • Load/store requires data access • Instruction fetch would have to stall for that cycle • Would cause a pipeline “bubble” • Hence, pipelined datapaths require separate instruction/data memories • Or separate instruction/data caches

8. Data Hazards • Instruction cannot execute in proper clock cycle because data that is needed is not yet available add $s0, $t0, $t1sub $t2, $s0, $t3

9. Simplified MIPS Pipelined Datapath

10. Forwarding (aka Bypassing) • Use result when it is computed • Don’t wait for it to be stored in a register • Requires extra connections in the datapath

11. Load-Use Data Hazard • Can’t always avoid stalls by forwarding • If value not computed when needed • Can’t forward backward in time!

12. Code Scheduling to Avoid Stalls • Reorder code to avoid use of load result in the next instruction: C code A = B + E, C= B + F lw $t1, 0($t0) lw $t2, 4($t0) add $t3, $t1, $t2 sw $t3, 12($t0) lw $t4, 8($t0) add $t5, $t1, $t4 sw $t5, 16($t0) lw $t1, 0($t0) lw $t2, 4($t0) lw $t4, 8($t0) add $t3, $t1, $t2 sw $t3, 12($t0) add $t5, $t1, $t4 sw $t5, 16($t0) stall stall

13. Control Hazards • Instruction cannot execute in proper clock cycle because the instruction that was fetched is not the one that is needed • Branch determines flow of control • Fetching next instruction depends on branch outcome • Pipeline can’t always fetch correct instruction • In MIPS pipeline • Need to compare registers and compute target early in the pipeline • Add hardware to do it in ID stage

14. Stall on Branch • Wait until branch outcome determined before fetching next instruction

15. Branch Instruction Data/Control Flow 0 Add Add 1 4 Shift left 2 PCSrc ALUOp Branch MemRead Instr[31-26] Control Unit MemtoReg MemWrite ALUSrc RegWrite RegDst ovf Instr[25-21] Read Addr 1 Instruction Memory Read Data 1 Address Instr[20-16] Register File zero Read Addr 2 Data Memory Read Address PC Instr[31-0] 0 Read Data 1 ALU Write Addr Read Data 2 0 1 Write Data 0 Instr[15 -11] Write Data 1 Instr[15-0] Sign Extend ALU control 16 32 Instr[5-0]

16. Branch Prediction • Correct branch prediction is very important and can produce substantial performance improvements. • static prediction • dynamic prediction • To take full advantage of branch prediction, we can have the instructions not only fetched but also begin execution. This is known as speculative execution

17. MIPS with Predict Not Taken Prediction correct Prediction incorrect

18. Branches • Branch instructions can dramatically affect pipeline performance. Control operations are very frequent in current programs. • 20% - 35% of the instructions executed are branches (conditional and unconditional). • 65% of the branches actually take the branch. • Conditional branches are much more frequent than unconditional (more than two times). More than 50% of conditional branches are taken.

19. Static Branch Prediction • Static prediction techniques do not take into consideration execution history. • Predict never taken (Motorola 68020): assumes that the branch is not taken. • Predict always taken: assumes that the branch is taken.

20. Dynamic Branch Prediction • Improve the accuracy of prediction by recording the history of conditional branches. • One-bit prediction scheme • is used in order to record if the last execution resulted in a branch taken or not. The system predicts the same behavior as for the last time. • Two-bit prediction scheme • with a two-bit scheme predictions can be made depending on the last two instances of execution.

21. One-Bit Prediction Scheme

22. Two-Bit Prediction Scheme

23. Branch History Table • History info. can be used not only to predict the outcome of a conditional branch but also to avoid recalculation of the target address. Together with bits used for prediction, the target address can be stored for later use in a branch history table. • Using D. B. P with history tables up to 90% of predictions can be correct. • Pentium,PowerPC620 use speculative execution with D.B.P based on a branch history table.

24. Branch History Table