Chapter Six

1. Chapter Six Pipelining: Dependencies & Hazards

2. Dependencies • Problem with starting next instruction before first is finished • dependencies that “go backward in time” are data hazards

3. Software Solution • Have compiler guarantee no hazards • Where do we insert the “nops” ? sub $2, $1, $3 and $12, $2, $5 or $13, $6, $2 add $14, $2, $2 sw $15, 100($2) • Problem: this really slows us down!

4. what if this $2 was $13? Forwarding • Use temporary results, don’t wait for them to be written • register file forwarding to handle read/write to same register • ALU forwarding

5. Forwarding Solution: if can take the inputs to the ALU from any pipeline register rather than just ID/EX, then can forward the proper data. Add multiplexors to the input of the ALU and with the proper controls, can run the pipeline at full speed in the presence of these data dependencies. assume only instructions to forward are the four R-format instructions: add, sub, and, or.

6. Forwarding

7. Forwarding values of the control lines for the ALU multiplexors that select either the register file values or one of the forwarded values.

8. Forwarding • Forwarding control is in the EX stage, because the ALU forwarding multiplexors are found in that stage. Must pass the operand register numbers from the ID stage via the ID/EX pipeline register to determine whether to forward values. Already have the rt filed (bits 20-16). Before forwarding, the ID/EX register had no need to include space to hold the rs field. Thus must add the rs bits (25-21) to ID/EX.

9. Forwarding When an instruction tries to read a register in its EX stage that an earlier instruction intends to write in its WB stage, we actually need the values as inputs to the ALU. Notation: need to name the fields of the pipeline registers precisely. ID/EX.RegisterRs refers to the number of one register whose value is found in the pipeline register ID/EX. The one from the first read port of the register file. The first part of the name, to the left of the period, is the name of the pipeline register The second part of the name is the name of the field in that register.

10. Forwarding The hazard conditions are:1a. EX/MEM.RegisterRd = ID/EX.RegisterRs1b. EX/MEM.RegisterRd = ID/EX.RegisterRt2a. MEM/WB.RegisterRd = ID/EX.RegisterRs2b. MEM/WB.RegisterRd = ID/EX.RegisterRt The first hazard in the sequence below is on register $2, between the result of SUB $2, $1, $3 and the first read operand of AND $12, $2, $5. sub $2, $1, $3 and $12, $2, $5 or $13, $6, $2 add $14, $2, $2 sw $15, 100($2) Detect hazard when the AND instruction is in the EX stage and the prior instruction is in the MEM stage. Thus, this is hazard 1a: 1a. EX/MEM.RegisterRd = ID/EX.RegisterRs = $2

11. Forwarding Problem: some instructions do not write registers. The policy is inaccurate. Solution: check to see if the RegWrite signal will be active: examine the WB control field of the pipeline register during the EX and MEM stages. Also, $0 will always be 0. Don’t forward result if the destination register is $0.

12. Forwarding The conditions for detecting hazards and the control signals to resolve them:1. EX hazard:if (EX/MEM.RegWrite and (Ex/MEM.RegisterRd <> 0) and (EX/MEM.RegisterRd = ID/EX.RegisterRs))then ForwardA = 10if (EX/MEM.RegWrite and (Ex/MEM.RegisterRd <> 0) and (EX/MEM.RegisterRd = ID/EX.RegisterRt))then ForwardB = 10This case forwards the result from the previous instruction to either input of the ALU.

13. Forwarding The conditions for detecting hazards and the control signals to resolve them:2. MEM hazard:if (MEM/WB.RegWrite and (MEM/WB.RegisterRd <> 0) and (MEM/WB.RegisterRd =ID/EX.RegisterRs))then ForwardA = 01if (MEM/WB.RegWrite and (MEM/WB.RegisterRd <> 0) and (MEM/WB.RegisterRd =ID/EX.RegisterRt))then ForwardB = 01

14. Forwarding There is no hazard in the WB stage. Assume that the register file supplies the correct result if the instruction in the ID stage reads the same register written by the instruction in the WB stage. In this case forwarding occurs within the register file.

15. Forwarding Complication: potential data hazards between the result of the instruction in the WB stage, the result of the instruction in the MEM stage and the source operand of the instruction in the ALU stage. Example: when summing a vector of numbers in a single register, a sequence of instructions will all read and write to the same register:add $1, $1, $2;add $1, $1, $3;add $1, $1, $4; Result is forwarded from the MEM stage because the result in the MEM stage is the more recent result.

16. Forwarding if (MEM/WB.RegWrite and (MEM/WB.RegisterRd <> 0) and (Ex/MEM.RegisterRd <> ID/EX.RegisterRs) and (MEM/WB.RegisterRd =ID/EX.RegisterRs))then ForwardA = 01if (MEM/WB.RegWrite and (MEM/WB.RegisterRd <> 0) and (Ex/MEM.RegisterRd <> ID/EX.RegisterRt) and (MEM/WB.RegisterRd =ID/EX.RegisterRt))then ForwardB = 01

17. Forwarding • Hardware

18. Forwarding • Example: sub $2, $1, $3 and $12, $2, $5 or $13, $6, $2 add $14, $2, $2 sw $15, 100($2) • See following slides

19. Forwarding

20. Forwarding

21. Forwarding

22. Forwarding

23. Forwarding • Since hazards are officially defined with respect to a particular hardware datapath, instruction sequences with result dependencies are no longer considered hazards when forwarding resolves conflicts. Thus name is forwarding unit not hazard forwarding unit. • Alternative solution: determine the control of the multiplexors on the ALU inputs during the ID stage, setting those values in new control fields of the ID/EX pipeline register. • Hardware may be faster since the time to select the ALU inputs is likely to be on the critical path for the EX stage. • The nop operation is represented by all 0’s. Represents sll $0, $0, 0. Does nothing to register $0 which can’t be changed anyway.

24. Forwarding • Forwarding can also help with hazards when store instructions are dependent on other instructions. • Since stores use just one register value during the MEM stage, forwarding is easy. • Problem: loads immediately followed by stores. Need to add more forwarding hardware to make memory-to-memory copies run faster. • Redraw figure 6.17 replacing sub and and instructions with lw and sw. Can avoid a stall since the data exists in the MEM/WB register of a load instruction in time for its use in the MEM stage of a store instruction. • Need to add forwarding into the memory access stage for this option. See exercise 6.20.

25. Forwarding • The signed-immediate input to the ALU, needed by loads and stores, is missing from the datapath figure show earlier. • central control decides between registers and immediate • the forwarding unit chooses the pipeline register for a register input to the ALU • the easiest solution is to add a 2:1 multiplexor that chooses between the ForwardB multiplexor output and the signed immediate. • See next slide. Note that this solution differs from chapter 5, where the mutiplexor controlled by line ALUSrcB was expanded to include the immediate input. • Solution also solves store forwarding by connecting the forwarding multiplexor output-containing store data in this case-to the EX/MEM pipeline register.

26. Forwarding

27. Data Hazards and Stalls • Load word can still cause a hazard: • an instruction tries to read a register following a load instruction that writes to the same register. • Thus, we need a hazard detection unit to “stall” the load instruction

28. Data Hazards and Stalls: hazard detection unit. •  Operates during the ID stage so that it can insert the stall between the load and its use. • Check for load instructions. Control for hazard detection unit is:if (ID/EX.MemRead and ((ID/EX.RegisterRt = IF/ID.RegisterRs) or (ID/EX.RegisterRt = IF/ID.RegisterRt)then stall the pipeline • First line tests to see if the instruction is a load: the only instruction that reads data memory is a load. • Next two lines check to see if the destination register field of the load in the EX stage matches either source register of the instruction in the ID stage. • If condition holds, stall 1 instruction. • After stall, the forwarding logic can handle the dependency and execution proceeds. • If there were no forwarding, then the instructions in figure 6.44 would need another stall.

29. Data Hazards and Stalls: hazard detection unit. •  If the instruction in the ID stage is stalled, then the instruction in the IF stage must also be stalled, o.w. would lose the fetched instruction. • To prevent these two instructions from making progress: prevent the PC register and the IF/ID pipeline register from changing. • If these registers are preserved, the instruction in the IF stage will continue to be read using the same PC • Also, the registers in the ID stage will continue to be read using the same instruction fields in the IF/ID pipeline register. • To stall, need same effect as inserting nop instructions. But this instruction must begin in the EX pipeline stage.

30. Data Hazards and Stalls: hazard detection unit. •  Must deassert all 9 control lines (set to 0) in the EX, MEM, and WB stages. Creates a “do nothing” instruction. • Identify the hazard in the ID stage. Then can insert a bubble into the pipeline by changing the EX,MEM, and WB control fields of the ID/EX pipeline register to 0. • These control values are percolated forward at each clock cycle.

31. Data Hazards and Stalls: hazard detection unit. •  Example: • lw $2, 20($1) and $4, $2, $5 or $8, $2, $6 add $9, $4, $2 slt $1, $6, $7 • Hazard forces the AND and OR instructions to repeat in clock cycle 4 what they did in clock cycle 3. • is refetched from instruction memory. • Effect is to stretch the time of the and and or instructions and delay the fetch of the add instruction.

32. Data Hazards and Stalls

33. Data Hazards and Stalls • Highlights the pipeline connections for both the hazard detection unit and the forwarding unit. • The forwarding unit controls the ALU multiplexors to replace the value from a general-purpose register with the value from the proper pipeline register. • The hazard detection unit controls the writing of the PC and IF/ID register plus the multiplexor that chooses between the real control values and all 0’s.

34. Data Hazards and Stalls

35. Data Hazards and Stalls

36. Data Hazards and Stalls

37. Data Hazards and Stalls

38. Data Hazards and Stalls

39. Data Hazards and Stalls

40. Branch Hazards • When we decide to branch, other instructions are in the pipeline! • We are predicting “branch not taken” • need to add hardware for flushing instructions if we are wrong

41. Flushing Instructions

42. Improving Performance • Try and avoid stalls! E.g., reorder these instructions: lw $t0, 0($t1) lw $t2, 4($t1) sw $t2, 0($t1) sw $t0, 4($t1) • Add a “branch delay slot” • the next instruction after a branch is always executed • rely on compiler to “fill” the slot with something useful • Superscalar: start more than one instruction in the same cycle

43. Dynamic Scheduling • The hardware performs the “scheduling” • hardware tries to find instructions to execute • out of order execution is possible • speculative execution and dynamic branch prediction • All modern processors are very complicated • DEC Alpha 21264: 9 stage pipeline, 6 instruction issue • PowerPC and Pentium: branch history table • Compiler technology important • This class has given you the background you need to learn more • Video: An Overview of Intel’s Pentium Processor (available from University Video Communications)