October 24nd, 2007 Majd F. Sakr msakr@qatar.cmu qatar.cmu/~msakr/15447-f07/

1. CS-447– Computer Architecture M,W 10-11:20amLecture 15Pipelining (3) October 24nd, 2007 Majd F. Sakrmsakr@qatar.cmu.edu www.qatar.cmu.edu/~msakr/15447-f07/

2. Can Pipelining Get Us Into Trouble? • Yes:Pipeline Hazards • structural hazards: attempt to use the same resource by two different instructions at the same time • data hazards: attempt to use data before it is ready • instruction source operands are produced by a prior instruction still in the pipeline • load instruction followed immediately by an ALU instruction that uses the load operand as a source value • control hazards: attempt to make a decision before condition has been evaluated • branch instructions • Can always resolve hazards by waiting • pipeline control must detect the hazard • take action (or delay action) to resolve hazards

3. Structural Hazard • Attempt to use same hardware for two different things at the same time. • Solution 1: Wait • Must detect hazard • Must have mechanism to stall • Solution 2: Throw more hardware at the problem

4. Reading data from memory Mem Mem Mem Mem Mem Reg Reg Reg Reg Reg Reg Reg Reg Reg Reg Mem Mem Mem Mem Mem ALU ALU ALU ALU ALU Reading instruction from memory A Single Memory Would Be a Structural Hazard Time (clock cycles) lw I n s t r. O r d e r Inst 1 Inst 2 Inst 3 Inst 4

5. DM DM DM DM DM Reg Reg Reg Reg Reg Reg Reg Reg Reg Reg IM IM IM IM IM ALU ALU ALU ALU ALU How About Register File Access? Time (clock cycles) Can fix register file access hazard by doing reads in the second half of the cycle and writes in the first half. add r1, I n s t r. O r d e r Inst 1 Inst 2 add r2,r1, Inst 4 Potential read before write data hazard

6. Three Generic Data Hazards • Read After Write (RAW)InstrJ tries to read operand before InstrI writes it • Caused by a “Data Dependence” (in compiler nomenclature). This hazard results from an actual need for communication. I: add r1,r2,r3 J: sub r4,r1,r3

7. I: sub r4,r1,r3 J: add r1,r2,r3 K: mul r6,r1,r7 Three Generic Data Hazards • Write After Read (WAR)InstrJ writes operand before InstrI reads it • Called an “anti-dependence” by compiler writers.This results from reuse of the name “r1”. • Can’t happen in MIPS 5 stage pipeline because: • All instructions take 5 stages, and • Reads are always in stage 2, and • Writes are always in stage 5

8. I: sub r1,r4,r3 J: add r1,r2,r3 K: mul r6,r1,r7 Three Generic Data Hazards Write After Write (WAW)InstrJ writes operand before InstrI writes it. • Called an “output dependence” by compiler writersThis also results from the reuse of name “r1”. • Can’t happen in MIPS 5 stage pipeline because: • All instructions take 5 stages, and • Writes are always in stage 5

9. DM DM DM DM DM Reg Reg Reg Reg Reg Reg Reg Reg Reg Reg IM IM IM IM IM ALU ALU ALU ALU ALU Register Usage Can Cause Data Hazards • Dependencies backward in time cause hazards add r1,r2,r3 I n s t r. O r d e r sub r4,r1,r5 and r6,r1,r7 or r8, r1, r9 xor r4,r1,r5 Which are read before write data hazards?

10. DM DM DM DM DM Reg Reg Reg Reg Reg Reg Reg Reg Reg Reg IM IM IM IM IM ALU ALU ALU ALU ALU Loads Can Cause Data Hazards • Dependencies backward in time cause hazards lw r1,100(r2) I n s t r. O r d e r sub r4,r1,r5 and r6,r1,r7 or r8, r1, r9 xor r4,r1,r5 Load-use data hazard

11. DM DM DM Reg Reg Reg Reg Reg Reg stall IM IM IM ALU ALU ALU stall sub r4,r1,r5 and r6,r1,r7 One Way to “Fix” a Data Hazard Can fix data hazard by waiting – stall – but affects throughput add r1,r2,r3 I n s t r. O r d e r

12. DM DM DM DM DM Reg Reg Reg Reg Reg Reg Reg Reg Reg Reg IM IM IM IM IM ALU ALU ALU ALU ALU Another Way to “Fix” a Data Hazard Can fix data hazard by forwarding results as soon as they are available to where they are needed. add r1,r2,r3 I n s t r. O r d e r sub r4,r1,r5 and r6,r1,r7 or r8, r1, r9 xor r4,r1,r5

13. DM DM DM DM DM Reg Reg Reg Reg Reg Reg Reg Reg Reg Reg IM IM IM IM IM ALU ALU ALU ALU ALU Forwarding with Load-use Data Hazards • Will still need one stall cycle even with forwarding lw r1,100(r2) I n s t r. O r d e r sub r4,r1,r5 and r6,r1,r7 or r8, r1, r9 xor r4,r1,r5

14. Control Hazards • Caused by delay between the fetching of instructions and decisions about changes in control flow • Branches • Jumps

15. DM DM DM Reg Reg Reg Reg Reg Reg IM IM IM IM ALU ALU ALU ALU beq DM Reg Reg Branch Instructions Cause Control Hazards • Dependencies backward in time cause hazards I n s t r. O r d e r lw Inst 3 Inst 4

16. DM DM Reg Reg Reg Reg IM IM IM ALU ALU ALU stall stall stall lw DM Reg Inst 3 One Way to “Fix” a Control Hazard beq Can fix branch hazard by waiting – stall – but affects throughput I n s t r. O r d e r

17. 1 ID/EX 0 EX/MEM IF/ID Control Add MEM/WB Add 4 Shift left 2 Read Addr 1 Instruction Memory Data Memory Register File Read Data 1 Read Addr 2 Read Address PC Read Data Address 1 Write Addr ALU Read Data 2 0 Write Data 0 Write Data 1 Sign Extend 16 32 Pipeline Control Path Modifications • All control signals can be determined during Decode • and held in the state registers between pipeline stages

18. Speed Up Equation for Pipelining For simple RISC pipeline, CPI = 1:

19. Performance • Speed Up  Pipeline Depth; if ideal CPI is 1, then: • Time is measure of performance: latency or throughput • CPI Law: CPU time = Seconds = Instructions x Cycles x Seconds Program Program InstructionCycle

20. DM Reg Reg IM IM ALU ALU DM2 DM1 Reg Reg Other Pipeline Structures Are Possible • What about (slow) multiply operation? • let it take two cycles MUL • What if the data memory access is twice as slow as the instruction memory? • make the clock twice as slow or … • let data memory access take two cycles (and keep the same clock rate)

21. Reg EX DM Reg Reg IM IM ALU ALU Sample Pipeline Alternatives (for ARM ISA) • ARM7(3-stage pipeline) • StrongARM-1(5-stage pipeline) • XScale(7-stage pipeline) PC update IM access decode reg access ALU op DM access shift/rotate commit result (write back) Reg DM2 IM1 DM1 IM2 Reg SHFT PC update BTB access start IM access decode reg 1 access DM write reg write ALU op start DM access exception shift/rotate reg 2 access IM access

22. Summary • All modern day processors use pipelining • Pipelining doesn’t help latency of single task, it helps throughput of entire workload • Multiple tasks operating simultaneously using different resources • Potential speedup = Number of pipe stages • Pipeline rate limited by slowest pipeline stage • Unbalanced lengths of pipe stages reduces speedup • Time to “fill” pipeline and time to “drain” it reduces speedup • Must detect and resolve hazards • Stalling negatively affects throughput