CS 6461: Computer Architecture Instruction Level Parallelism

1. CS 6461: Computer ArchitectureInstruction Level Parallelism Instructor: M. Lancaster Corresponding to Hennessey and Patterson Fifth Edition Section 3.1

2. Instruction Level Parallelism • Almost all processors since 1985 use pipelining to overlap the execution of instructions and improve performance. This potential overlap among instructions is called instruction level parallelism • First introduced in the IBM Stretch (Model 7030) in about 1959 • Later the CDC 6600 incorporated pipelining and the use of multiple functional units • The Intel i486 was the first pipelined implementation of the IA32 architecture Instruction Level Parallelism

3. Instruction Level Parallelism • Instruction level parallel processing is the concurrent processing of multiple instructions • Difficult to achieve within a basic code block • Typical MIPS programs have a dynamic branch frequency of between 15% and 25% • That is, between three and six instructions execute between a pair of branches, and data hazards usually exist within these instructions as they are likely to be dependent • Given basic code block size in number of instructions, ILP must be exploited across multiple blocks Instruction Level Parallelism

4. Instruction Level Parallelism • The current trend is toward very deep pipelines, increasing from a depth of < 10 to > 20. • With more stages, each stage can be smaller, more simple and provide less gate delay, therefore very high clock rates are possible. Instruction Level Parallelism

5. Loop Level ParallelismExploitation among Iterations of a Loop • Loop adding two 1000 element arrays • Codefor (i=1; i<= 1000; i=i+1) x[i] = x[i] + y[i]; • If we look at the generated code, within a loop there may be little opportunity for overlap of instructions, but each iteration of the loop can overlap with any other iteration Instruction Level Parallelism

6. Concepts and ChallengesApproaches to Exploiting ILP • Two major approaches • Dynamic – these approaches depend upon the hardware to locate the parallelism • Static – fixed solutions generated by the compiler, and thus bound at compile time • These approaches are not totally disjoint, some requiring both • Limitations are imposed by data and control hazards Instruction Level Parallelism

7. Features Limiting Exploitation of Parallelism • Program features • Instruction sequences • Processor features • Pipeline stages and their functions • Interrelationships • How do program properties limit performance? Under what circumstances? Instruction Level Parallelism

8. Approaches to Exploiting ILPDynamic Approach • Hardware intensive approach • Dominate desktop and server markets • Pentium III, 4, Athlon • MIPS R10000/12000 • Sun UltraSPARC III • PowerPC 603, G3, G4 • Alpha 21264 Instruction Level Parallelism

9. Approaches to Exploiting ILPStatic Approach • Compiler intensive approach • Embedded market and IA-64 Instruction Level Parallelism

10. Terminology and Ideas • Cycles Per Instruction • Pipeline CPI = Ideal Pipeline CPI + Structural Stalls + Data Hazard Stalls + Control Stalls • Ideal Pipeline CPI is the max that we can achieve in a given architecture. Stalls and/or their impacts must be minimized. • During 1980s CPI =1 was a target objective for single chip microprocessors • 1990’s objective: reduce CPI below 1 • Scalar processors are pipelined processors that are designed to fetch and issue at most one instruction every machine cycle • Superscalar processors are those that are designed to fetch and issue multiple instructions every machine cycle Instruction Level Parallelism

11. Approaches to Exploiting ILP That We Will Explore Instruction Level Parallelism

12. Approaches to Exploiting ILPReview of Terminology • Instruction issue: • The process of letting an instruction move from the instruction decode phase (ID) into the instruction execution (EX) phase • Interlock (pipeline interlock, instruction interlock) is the resolution of pipeline hazards via hardware. Pipeline interlock hardware must detect all pipeline hazards and ensure that all dependencies are satisfied Instruction Level Parallelism

13. Data Dependencies and Hazards • How much parallelism exists in a program and how it can be exploited • If two instructions are parallel, they can execute simultaneously in a pipeline without causing any stalls (assuming no structural hazards exist) • There are no dependencies in parallel instructions • If two instructions are not parallel and must be executed in order, they may often be partially overlapped. Instruction Level Parallelism

14. Pipeline Hazards • Hazards make it necessary to stall the pipeline. • Some instructions in the pipeline are allowed to proceed while others are delayed • For this example pipeline approach, when an instruction is stalled, all instructions further back in the pipeline are also stalled • No new instructions are fetched during the stall • Instructions issued earlier in the pipeline must continue Instruction Level Parallelism

15. Data Dependencies and Hazards • Data Dependences – an instruction j is data dependent on instruction i if either of the following holds • Instruction i produces a result that may be used by instruction j • Instruction j is data dependent on instruction k, and instruction k is data dependent on instruction i – that is, one instruction is dependent on another if there exists a chain of dependencies of the first type between two instructions. Instruction Level Parallelism

16. Data Dependencies and Hazards • Data Dependences – • Code ExampleLOOP: L.D F0,0(R1) ;F0=array element ADD.D F4,F0,F2 ;add scalar in F2 S.D F4,0(R1) ;store result DADDUI R1,R1,#-8 ;decrement pointer 8 BNE R1,R2,LOOP; • The above dependencies are in floating point data for the first two arrows, and integer data in the last two instructions Instruction Level Parallelism

17. Data Dependencies and Hazards • Data Dependences – • Arrows show where order of instructions must be preserved • If two instructions are dependent, they cannot be simultaneously executed or be completely overlapped Instruction Level Parallelism

18. Data Dependencies and Hazards • Dependencies are properties of programs • Whether a given dependence results in an actual hazard being detected and whether that hazard actually causes a stall are properties of the pipeline organization Instruction Level Parallelism

19. Data Dependencies and Hazards • Hazard created – • Code Example DADDUI R1,R1,#-8 ;decrement pointer 8 BNE R1,R2,LOOP ; • When the branch test is moved from EX to ID stage • If test stayed in ID, dependence would not cause a stall (Branch delay would still be two cycles however) Instruction Level Parallelism

20. Data Dependencies and Hazards Branch destination and test known at end of second cycle of execution Branch destination and test known at end of third cycle of execution Instruction Level Parallelism

21. Data Dependencies and Hazards • Presence of dependence indicates a potential for a hazard, but the actual hazard and the length of any stall is a property of the pipeline. • Data dependence • Indicates possibility of stall • Determines the order in which results are calculated • Sets an upper bound on how much parallelism can be possibly exploited. • We will focus on overcoming these limitation Instruction Level Parallelism

22. Overcoming Dependences • Two Ways • Maintain dependence but avoid the hazard • Schedule the code dynamically • Transform the code Instruction Level Parallelism

23. Difficulty in Detecting Dependences • A data value may flow between instructions either through registers or through memory locations • Therefore, detection is not always straightforward • For instructions referring to memory, the register dependences are easy to detect • Suppose however we have R4 = 20 and R6 = 100 and we use 100(R4) and 20(R6) • Suppose we have incremented R4 in an instruction between two references (say 20(R4) ) that look identical Instruction Level Parallelism

24. Name Dependences; Two Categories • Two instructions use the same register or memory location, called a name, but there is actually no flow of data between the instructions associated with that name. In cases where i precedes j. • 1. An antidependence between instructions i and j occurs when instruction j writes a register or memory location that instruction i reads. The original ordering must be preserved • 2. An output dependence occurs when instruction i and instruction j write the same register or memory location, the order again must be preserved Instruction Level Parallelism

25. Name Dependences; Two Categories • 1. An antidependence • i DADD R1,R2.#-8 • j DADD R2,R5,0 • 2. An output dependence • i DADD R1,R2.#-8 • j DADD R1,R4,#10 Instruction Level Parallelism

26. Name Dependences • Not true data dependencies, and therefore we could execute them simultaneously or reorder them if the name (register or memory location) used in the instructions is changed so that the instructions do not conflict • Register renaming is easier • i DADD R1,R2,#-8 • j DADD R2,R4,#10 i DADD R1,R2,#-8 • j DADD R5,R4,#10 Instruction Level Parallelism

27. Data Hazards • A hazard is created whenever there is a dependence between instructions, and they are close enough that the overlap caused by pipelining or other reordering of instructions would change the order of access to the operand involved in the dependence. • We must preserve program order; the order the instructions would execute if executed in a non-pipelined system • However, program order only need be maintained where it affects the outcome of the program Instruction Level Parallelism

28. Data Hazards – Three Types • Two instructions i and j, with i occurring before j in program order, possible hazards are: • RAW (read after write) – j tries to read a source before i writes it, so j incorrectly gets the old value • The most common type • Program order must be preserved • In a simple common static pipeline a load instruction followed by an integer ALU instruction that directly uses the load result will lead to a RAW hazard Instruction Level Parallelism

29. Data Hazards – Three Types • Second type: • WAW (write after write) – j tries to write an operand before it is written by i, with the writes ending up in the wrong order, leaving value written by i • Output dependence • Present in pipelines that write in more than one pipe or allow an instruction to proceed even when a previous instruction is stalled • In the classic example, WB stage is used for write back, this class of hazards avoided. • If reordering of instructions is allowed this is a possible hazard • Suppose an integer instruction writes to a register after a floating point instruction does Instruction Level Parallelism

30. Data Hazards – Three Types • Third type: • WAR (write after read) – j tries to write an operand before it is read by i, so i incorrectly gets the new value. • Antidependence • Cannot occur in most static pipelines – note that reads are early in ID and writes late in WB Instruction Level Parallelism

31. Control Dependencies • Determines ordering of instruction, i with respect to a branch instruction so that the instruction i is executed in the correct program order and only when it should be. • Example • if p1 { S1;};if p2 { S2;} Instruction Level Parallelism

32. Control Dependencies • Example • if p1 { S1;};if p2 { S2;} • S1 is control dependent on p1 and S2 is control dependent on P2 but not on P1 Instruction Level Parallelism

33. Control Dependencies • Two constraints imposed • An instruction that is control dependent on a branch cannot be moved before the branch so that its execution is no longer controlled by the branch. For example we cannot take a statement from the then portion of an if statement and move it before the if statement. • An instruction that is not control dependent on a branch cannot be moved after the branch so that the execution is controlled by the branch. For example, we cannot take a statement before the if and move it into the then portion if p1 { S1;};if p2 { S2;} Instruction Level Parallelism

34. Control Dependencies • Two properties of our simple pipeline preserve control dependencies • Instructions execute in program order • Detection of control or branch hazards ensures that an instruction that is control dependent on a branch is not executed until the branch direction is known • We can introduce instructions that should not have been executed (violating control dependences) if we can do so without affecting the correctness of the program Instruction Level Parallelism

35. Control Dependencies are Really… • Not the issue; Really the issue is the preservation of • Exception behavior • Data flow Instruction Level Parallelism

36. Preserving Exception Behavior • Preserving exception behavior means that any changes in the ordering of instruction execution must not change how exceptions are raised in the program • We may relax this rule and say that reordering of instruction execution must not cause any new exceptions DADDU R2,R3,R4 BEQZ R2, L1 LW R1,0(R2) ;Could cause illegal mem acc L1: … • In the above, if we do not maintain the data dependence of R2, we may change the program. If we ignore the control dependency and move the load instruction before the branch, the load instruction may cause a memory protection exception • There is no visible data dependence that prevents this interchange, only control dependence Instruction Level Parallelism

37. Preserving Exception Behavior • To allow reordering of these instructions (which as we said preserves data dependence) we would like to just ignore the exception. Instruction Level Parallelism

38. Preserving Data Flow • This means preserving the actual flow of data values between instructions that produce results and those that consume them. • Branches make data flow dynamic, since they allow the source of data for a given instruction to come from many points Instruction Level Parallelism

39. Preserving Data Flow • Example DADDU R1,R2,R3 BEQZ R4,L DSUBU R1,R5,R6L: … OR R7,R1,R8 ; depends on branch taken • Cannot move DSUBU above branch • By preserving the control dependence of the OR on the branch we prevent an illegal change to the data flow Instruction Level Parallelism

40. Preserving Data Flow • Sometimes violating the control dependence cannot affect either the exception behavior or the data flow DADDU R1,R2,R3 BEQZ R1,skip DSUBU R4,R5,R6 DADDU R5,R4,R9skip: OR R7,R1,R8 ; suppose R4 not used after here • If R4 unused after this point, changing the value of R4 just before the branch would not affect data flow • If R4 were dead and DSUBU could not generate an exception* we could move the DSUBU instruction before the branch • This is called speculation since compiler is betting on branch outcome Instruction Level Parallelism

41. Control Dependence Again • Control dependence in the simple pipeline is preserved by implementing control and hazard detection that can cause control stalls • Can be eliminated by a variety of hardware techniques • Delayed branches can reduce stalls arising from control hazards, but requires that the compiler preserve data flow Instruction Level Parallelism