Instruction Level Parallelism

Instruction Level Parallelism • Scalar-processors • the model so far • SuperScalar • multiple execution units in parallel • VLIW • multiple instructions read in parallel

Scalar Processors • T = Nq * CPI * Ct • The time to perform a task • Nq, number of instruction, CPI cycles/instruction, Ct cycle-time • Pipeline • CPI = 1 • Ct determined by critical path • But: • Floating point operations slow in software • Even in hardware (FPU) takes several cycles WHY NOT USE SEVERAL FLOATING POINT UNITS?

SuperScalar Processors 1-cycle issue completion ALU IF DE PFU 1 DM WB …. PFU n m-cycles Each unit may take several cycles for finish

Instruction VS Machine Parallelism • Instruction Parallelism • Average nr of instructions that can be executed in parallel • Depends on; • “true dependencies” • Branches in relation to other instructions • Machine Parallelism • The ability of the hardware to utilize instruction parallelism • Depends on; • Nr of instructions that can be fetched and executed each cycle • instruction memory bandwidth and instruction buffer (window) • available resources • The ability to spot instruction parallelism

Example 1 1) add $t0 $t1 $t2 2) addi $t0 $t0 1 3) sub $t3 $t1 $t2 4) subi $t3 $t3 1 dependent instruction lookahead or “prefetch” independent dependent 1) add $t0 $t1 $t2 2) addi $t0 $t0 1 3) sub $t3 $t1 $t2 4) subi $t3 $t3 1 Concurrentlyexecuted

Issue & Completion • Out of order issue, (starts “out of order”) • RAW hazards • WAR hazard (write after read) • Out of order completion, (finishes “out of order”) • WAW, Antidependence hazard (result overwritten) Issue Completion 1) add $t0 $t1 $t2 2) addi $t0 $t0 1 3) sub $t3 $t1 $t2 4) subi $t3 $t3 1 1) 3) - - 2-parallel execution units 4-stage pipeline 2) 4) - - - - 1) 3) 2) 4)

Tomasulo’s Algorithm A mul $r1 2 3 mul $r2 $r1 4 mul $r2 5 6 IF DE B DM WB IDLE A C IDLE B IDLE C ... ... ... ... ...

Instruction Issue mul $r1 2 3 2 A 3 mul $r1 2 3 mul $r2 $r1 4 mul $r2 5 6 IF DE B DM WB BUSY A $r1 A C IDLE B IDLE C ... ... ... ... ...

Instruction Issue mul $r1 2 3 2 A 3 mul $r2 A 4 mul $r1 2 3 mul $r2 $r1 4 mul $r2 5 6 A IF DE B DM WB 4 BUSY A $r1 A C WAIT B $r2 B IDLE C ... ... ... ... ...

Instruction Issue mul $r1 2 3 2 A 3 mul $r2 A 4 mul $r1 2 3 mul $r2 $r1 4 mul $r2 5 6 A IF DE B DM WB 4 mul $r2 5 6 BUSY A $r1 A C WAIT B $r2 C BUSY C ... ... ... ... ... Reg $r2 gets newer value

Clock until A and B finish mul $r1 2 3 2 A 3 mul $r2 6 4 mul $r1 2 3 mul $r2 $r1 4 mul $r2 5 6 6 IF DE B DM WB 4 mul $r2 5 6 IDLE A $r1 6 C BUSY B $r2 30 IDLE C ... ... ... ... ...

Clock until B finishes 2 A 3 mul $r2 6 4 mul $r1 2 3 mul $r2 $r1 4 mul $r2 5 6 6 IF DE B DM WB 2 IDLE A $r1 6 C IDLE B $r2 30 IDLE C ... ... ... ... ... NOT CHANGED!

SuperScalar Designs • 3-8 times faster than Scalar designs depending on • Instruction parallelism (upper bound) • Machine parallelism • Pros • Backward compatible (optimization is done at run time) • Cons • Complex hardware implementation • Not scaleable (Instruction Parallelism)

VLIW Why not let the compiler do the work? • Use a Very Long Instruction Word (VLIW) • Consisting of many instructions is parallel • Each time we read one VLIW instruction we actually issue all instructions contained in the VLIW instruction

VLIW Usually the bottleneck 32 IF DE EX DM WB VLIW instruction 32 IF DE EX DM WB 128 32 IF DE EX DM WB 32 IF DE EX DM WB

VLIW • Let the compiler can do the instruction issuing • Let it take it’s time we do this only once, ADVANCED • What if we change the architecture • Recompile the code • Could be done the first time you load a program • Only recompiled when architecture changed • We could also let the compiler know about • Cache configuration • Nr levels, line size, nr lines, replacement strategy, writeback/writethrough etc. Hot Research Area!

VLIW • Pros • We get high bandwidth to instruction memory • Cheap compared to SuperScalar • Not much extra hardware needed • More parallelism • We spot parallelism at a higher level (C, MODULA, JAVA?) • We can use advanced algorithms for optimization • New architectures can be utilized by recompilation • Cons • Software compatibility • It has not “HIT THE MARKET” (yet).

4 State Branch Prediction BRA NO BRA loop : A bne 100times loop B j loop 1 2 Predict Branch BRA NO BRA BRA We always predict BRA (1) in the inner loop, when exit we fail once and go to (2). Next time we still predict BRA (2) and go to (1) NO BRA Predict no branch BRA NO BRA

Branch Prediction • The 4-states are stored in 2 bits in the instruction cache together with the conditional Branch instruction • We predict the branch • We prefetch the predicted instructions • We issue these before we know if branch taken! • When predicting fails we abort issued instructions

Branch Prediction loop 1) 2) 3) Instructions 1) 2) and 3) are prefetched and may already be issued when we know the value of $r1, since $r1 might be waiting for some unit to finish predict branch taken bne $r1 loop In case of prediction failure we have to abort the issued instructions and start fetching 4) 5) and 6)

Multiple Branch Targets loop 1) 2) 3) Instructions 1) 2)3)4) 5) and 6) is prefetched and may already be issued when we know the value of $r1, since $r1 might be waiting for some unit to finish bne $r1 loop As soon as we know $r1 we abort the redundant instructions. VERY COMPLEX!!!

Instruction Level Parallelism