Dynamic Scheduling in Computer Systems

1. CENG 450Computer Systems and ArchitectureLecture 9 Amirali Baniasadi amirali@ece.uvic.ca

2. This Lecture • ILP • Scheduling

3. Dynamic Scheduling • Dynamic Scheduling: Hardware rearranges the order of instruction execution to reduce stalls • Disadvantages • Hardware much more complex • Key idea • Instructions execution in parallel (use available all executing units) • Allow instructions behind stall to proceed • Example • DIVD F0,F2,F4 • ADDD F10,F0,F8 • SUBD F12,F8,F14 • Out-of-order execution => out-of-order completion

4. Overview • In-order pipeline • 5 interlocked stages: IF, ID, EX, MEM, WB • Structural hazard: maximum of 1 instruction per stage • Unless stage is replicated (FP & integer EX) or “idle” (WB for stores) • Out-of-order pipeline • How does one instruction pass another without “killing”it? • Remember: only one instruction per-stage per-cycle • Must “buffer” instructions IF ID EX MEM WB

5. Instruction Buffer • Trick: instruction buffer (many names for this buffer) • Accumulate decoded instructions in buffer • Buffer sends instructions down rest of pipe out-of-order instruction buffer IF ID1 ID2 EX MEM WB

6. Scoreboard State/Steps instruction buffer • Confusion in community about which is which stage IF IS RO EX WB ID Structure Data Bus Registers EX EX EX Control/Status Scoreboard

7. Dynamic Scheduling: Scoreboard • Out-of-order execution divides ID stage: • 1.Issue—decode instructions, check for structural hazards • 2. Read Operands—wait until no data hazards, then read operands • Scoreboards allow instruction to execute whenever 1 & 2 hold, not waiting for prior instructions. • A scoreboard is a “data structure” that provides the information necessary for all pieces of the processor to work together. • Centralized control scheme • No bypassing • No elimination of WAR/WAW hazards • We will use In order issue, out of order execution, out of order commit ( also called completion) • First used in CDC6600.

8. Stages of Scoreboard Control • Issue—decode instructions & check for structural hazards (ID1) • If a functional unit for the instruction is free and no other active instruction has the same destination register (WAW), the scoreboard issues the instruction to the functional unit and updates its internal data structure. • If a structural or WAW hazard exists, then the instruction issue stalls, and no further instructions will issue until these hazards are cleared.

9. Stages of Scoreboard Control • Read Operands—wait until no data hazards, then read operands from registers (ID2) • A source operand is available if no earlier issued active instruction is going to write it, or if the register containing the operand is being written by a currently active functional unit. • When the source operands are available, the scoreboard tells the functional unit to proceed to read the operands from the registers and begin execution. • The scoreboard resolves RAW hazards dynamically in this step, and instructions may be sent into execution out of order.

10. Stages of Scoreboard Control • Execution—operate on operands (EX) • The functional unit begins execution upon receiving operands. When the result is ready, it notifies the scoreboard that it has completed execution. • Write result—finish execution (WB) • Once the scoreboard is aware that the functional unit has completed execution, the scoreboard checks for WAR hazards. If none, it writes results. If WAR, then it stalls the instruction. • Example: • DIVD F0, F2, F4 • ADDD F10, F0, F8 • SUBD F8, F8, F14 • Scoreboard would stall SUBD until ADDD reads operands

11. Scoreboard Data Structures • Instruction status • Which of 4 steps the instruction is in • Functional unit status • Busy Whether the unit is busy or not • Op Operation to perform in the unit (e.g., + or –) • Fi Destination register • Fj, Fk Source-register numbers • Qj, Qk Functional units producing source registers Fj, Fk • Rj, Rk ready bits for Fj, Fk • Register result status • Indicates which functional unit (if any) will write each register. • Blank when no pending instructions will write that register

12. Scoreboard Example LD F6, 34(R2) LD F2, 45(R3) MULT F0, F2, F4 SUBD F8, F6, F2 DIVD F10, F0, F6 ADDD F6, F8, F2 What are the hazards in this code? Latencies (clock cycles) LD 1 MULT 10 DIVD 40 ADDD, SUBD 2

13. Scoreboard Example

14. Scoreboard Example: Cycle 1 Issue LD #1 Shows in which cycle the operation occurred.

15. Scoreboard Example: Cycle 2 LD #2 can’t issue since integer unit is busy. MULT can’t issue because we require in-order issue.

16. Scoreboard Example: Cycle 3

17. Scoreboard Example: Cycle 4

18. Scoreboard Example: Cycle 5 Issue LD #2 since integer unit is now free

19. Scoreboard Example: Cycle 6 Issue MULT

20. Scoreboard Example: Cycle 7 MULT can’t read its operands (F2) because LD #2 hasn’t finished

21. Scoreboard Example: Cycle 8a DIVD issues. MULT and SUBD both waiting for F2

22. Scoreboard Example: Cycle 8b LD #2 writes F2

23. Scoreboard Example: Cycle 9 Now MULT and SUBD can both read F2 How can both instructions do this at the same time??

24. Scoreboard Example: Cycle 11 ADDD can’t start because Add unit is busy

25. Scoreboard Example: Cycle 12 SUBD finishes. DIVD waiting for F0

26. Scoreboard Example: Cycle 13 ADDD issues

27. Scoreboard Example: Cycle 14

28. Scoreboard Example: Cycle 15

29. Scoreboard Example: Cycle 16

30. Scoreboard Example: Cycle 17 ADDD can’t write because of DIVD RAW!

31. Scoreboard Example: Cycle 18 Nothing Happens!!

32. Scoreboard Example: Cycle 19 MULT completes execution

33. Scoreboard Example: Cycle 20 MULT writes

34. Scoreboard Example: Cycle 21 DIVD loads operands

35. Scoreboard Example: Cycle 22 Now ADDD can write since WAR removed

36. Scoreboard Example: Cycle 61 DIVD completes execution

37. Scoreboard Example: Cycle 62 DONE!!

38. Scoreboard • Operands for an instruction are read only when both operands are available in the register file • Scoreboard does not take advantage of forwarding • Instructions write to register file as soon as they are complete execution (assuming no WAR hazards) and do not wait for write slot • Reduced pipeline latency & benefits of forwarding • One additional cycle of latency as write result and read operand stages cannot overlap • Bus structure • Limited number of buses to register file represent structural hazards

39. Scoreboard • Limitations • No forwarding (RAW dependence handled through registers) • In-order issue for WAW/structural hazards limit scheduling flexibility • WAR stalls limit dynamic loop unrolling (no register unrolling) • Performance • 1.7X for FORTRAN programs • 2.5X for hand-coded assembly • Hardware • Scoreboard is cheap • Busses are not

40. Scoreboard Implications • Out-of-order completion => WAR, WAW hazards? • Solutions for WAR • Queue both the operation and copies of its operands • Read registers only during Read Operands stage • Solution for WAW & Structural Hazards • Must detect hazard: stall until the hazards are cleared • Need to have multiple instructions in execution phase • Multiple execution units or pipelined execution units • Scoreboard keeps track of dependencies, state or operations • Scoreboard replaces ID, EX, WB with 4 stages