Tomasulo Algorithm vs. Scoreboard: Dynamic Scheduling Comparison

1. CS252Graduate Computer ArchitectureLecture 7 Dynamic Scheduling 2: Precise Interrupts February 9th, 2010 John Kubiatowicz Electrical Engineering and Computer Sciences University of California, Berkeley http://www.eecs.berkeley.edu/~kubitron/cs252

2. FP Mult FP Mult FP Divide FP Add Integer Review: Scoreboard (CDC 6600) Registers Functional Units SCOREBOARD Memory CS252-S11, lecture 7

3. Review: Four Stages of Scoreboard Control • Issue—decode instructions & check for structural hazards • Instructions issued in program order (for hazard checking) • Don’t issue if structural hazard • Don’t issue if instruction is output dependent on any previously issued but uncompleted instruction (no WAW hazards) • Read operands—wait operands ready, then read them • All real dependencies (RAW hazards) resolved in this stage • No forwarding of data in this model! • Execution—operate on operands • The functional unit begins execution upon receiving operands. When the result is ready, it notifies the scoreboard that it has completed execution. • Write result—finish execution • Stall if WAR hazards with previous instructions:Example: DIVD F0,F2,F4 ADDD F10,F0,F8 SUBD F8,F8,F14CDC 6600 scoreboard would stall SUBD until ADDD reads operands CS252-S11, lecture 7

4. Review: Tomasulo Organization FP Registers From Mem FP Op Queue Load Buffers Load1 Load2 Load3 Load4 Load5 Load6 Store Buffers Add1 Add2 Add3 Mult1 Mult2 Reservation Stations To Mem FP adders FP multipliers Common Data Bus (CDB) CS252-S11, lecture 7

5. Review: Three Stages of Tomasulo Algorithm 1. Issue—get instruction from FP Op Queue If reservation station free (no structural hazard), control issues instr & sends operands (renames registers). 2. Execution—operate on operands (EX) When both operands ready then execute; if not ready, watch Common Data Bus for result 3. Write result—finish execution (WB) Write on Common Data Bus to all awaiting units; mark reservation station available • Normal data bus: data + destination (“go to” bus) • Common data bus: data + source (“come from” bus) • 64 bits of data + 4 bits of Functional Unit source address • Write if matches expected Functional Unit (produces result) • Does the broadcast CS252-S11, lecture 7

6. Review: Compare to Scoreboard Cycle 62 • Why take longer on scoreboard/6600? • Structural Hazards • Lack of forwarding CS252-S11, lecture 7

7. Review: Loop Example Cycle 9 • Dataflow graph constructed completely in hardware • Renaming detaches early iterations from registers CS252-S11, lecture 7

8. Explicit Register Renaming • Tomasulo provides Implicit Register Renaming • User registers renamed to reservation station tags • Explicit Register Renaming: • Use physical register file that is larger than number of registers specified by ISA • Keep a translation table: • ISA register => physical register mapping • When register is written, replace table entry with new register from freelist. • Physical register becomes free when not being used by any instructions in progress. • Pipeline can be exactly like “standard” DLX pipeline • IF, ID, EX, etc…. • Advantages: • Removes all WAR and WAW hazards • Like Tomasulo, good for allowing full out-of-order completion • Allows data to be fetched from a single register file • Makes speculative execution/precise interrupts easier: • All that needs to be “undone” for precise break pointis to undo the table mappings CS252-S11, lecture 7

9. FP Mult FP Mult FP Divide Registers Functional Units FP Add Integer SCOREBOARD Memory Question: Can we use explicit register renaming with scoreboard? Rename Table CS252-S11, lecture 7

10. Scoreboard Example • Initialized Rename Table CS252-S11, lecture 7

11. Renamed Scoreboard 1 • Each instruction allocates free register • Similar to single-assignment compiler transformation CS252-S11, lecture 7

12. Renamed Scoreboard 2 CS252-S11, lecture 7

13. Renamed Scoreboard 3 CS252-S11, lecture 7

14. Renamed Scoreboard 4 CS252-S11, lecture 7

15. Renamed Scoreboard 5 CS252-S11, lecture 7

16. Renamed Scoreboard 6 CS252-S11, lecture 7

17. Renamed Scoreboard 7 CS252-S11, lecture 7

18. Renamed Scoreboard 8 CS252-S11, lecture 7

19. Renamed Scoreboard 9 CS252-S11, lecture 7

20. WAR Hazard gone! Renamed Scoreboard 10 • Notice that P32 not listed in Rename Table • Still live. Must not be reallocated by accident CS252-S11, lecture 7

21. Renamed Scoreboard 11 CS252-S11, lecture 7

22. Renamed Scoreboard 12 CS252-S11, lecture 7

23. Renamed Scoreboard 13 CS252-S11, lecture 7

24. Renamed Scoreboard 14 CS252-S11, lecture 7

25. Renamed Scoreboard 15 CS252-S11, lecture 7

26. Renamed Scoreboard 16 CS252-S11, lecture 7

27. Renamed Scoreboard 17 CS252-S11, lecture 7

28. Renamed Scoreboard 18 CS252-S11, lecture 7

29. Explicit Renaming Support Includes: • Rapid access to a table of translations • A physical register file that has more registers than specified by the ISA • Ability to figure out which physical registers are free. • No free registers  stall on issue • Thus, register renaming doesn’t require reservation stations. However: • Many modern architectures use explicit register renaming + Tomasulo-like reservation stations to control execution. CS252-S11, lecture 7

30. Which features of an ISA limit the number of instructions in the pipeline? Which features of a program limit the number of instructions in the pipeline? How many instructions can be in the pipeline? Number of Registers Control transfers Out-of-order dispatch by itself does not provide any significant performance improvement ! CS252-S11, lecture 7

31. 2 1 3 4 5 6 How important is renaming?Consider execution without it latency 1 LD F2, 34(R2) 1 2 LD F4, 45(R3) long 3 MULTD F6, F4, F2 3 4 SUBD F8, F2, F2 1 5 DIVD F4, F2, F8 4 6 ADDD F10, F6, F4 1 In-order: 1 (2,1) . . . . . . 2 3 4 43 5 . . . 5 6 6 Out-of-order: 1 (2,1) 4 4 . . . . 2 3 . . 3 5 . . . 5 6 6 Out-of-order execution did not allow any significant improvement! CS252-S11, lecture 7

32. Little’s Law Throughput (T) = Number in Flight (N) / Latency (L) WB Execution Issue • Example: • 4 floating point registers • 8 cycles per floating point operation • maximum of ½ issue per cycle without renaming! CS252-S11, lecture 7

33. 2 1 3 4 5 6 Instruction-level Parallelism via Renaming latency 1 LD F2, 34(R2) 1 2 LD F4, 45(R3) long 3 MULTD F6, F4, F2 3 4 SUBD F8, F2, F2 1 5 DIVD F4’, F2, F8 4 6 ADDD F10, F6, F4’1 X In-order: 1 (2,1) . . . . . . 2 3 4 43 5 . . . 5 6 6 Out-of-order: 1 (2,1) 4 4 5 . . . 2 (3,5) 3 6 6 • Any antidependence can be eliminated by renaming. • (renaming  additional storage) • Can be done either in Software or Hardware CS252-S11, lecture 7

34. Administrative • Midterm I: Wednesday 3/14Location: 320 Soda HallTIME: 2:30-5:30 • Can have 1 sheet of 8½x11 handwritten notes – both sides • No microfiche of the book! • This info is on the Lecture page • Meet at LaVal’s afterwards for Pizza and Beverages • Great way for me to get to know you better • I’ll Buy! • Don’t quite have new set of papers up. Will do that by tomorrow at the latest…. CS252-S11, lecture 7

35. B A + * / + X(0) Y X Data-Flow Architectures • Basic Idea: Hardware respresents direct encoding of compiler dataflow graphs: • Data flows along arcs in“Tokens”. • When two tokens arrive atcompute box, box “fires” andproduces new token. • Split operations produce copiesof tokens Input: a,b y:= (a+b)/x x:= (a*(a+b))+b output: y,x CS252-S11, lecture 7

36. Operation Unit 0 Instruction Cell Operation Unit m-1 Instruction Instruction Cell 0 Operation Packet Operand 1 Data Packets Instruction Cell 1 Operand 2 Memory Instruction Cell n-1 Paper by Dennis and Misunas “Reservation Station?” CS252-S11, lecture 7

37. What about Precise Exceptions/Interrupts? • Both Scoreboard and Tomasulo have: • In-order issue, out-of-order execution, out-of-order completion • Recall: An interrupt or exception is precise if there is a single instruction for which: • All instructions before that have committed their state • No following instructions (including the interrupting instruction) have modified any state. • Need way to resynchronize execution with instruction stream (I.e. with issue-order) • Easiest way is with in-order completion (i.e. reorder buffer) • Other Techniques (Smith paper): Future File, History Buffer CS252-S11, lecture 7

38. Commit Point Inst. Mem Decode Data Mem + Exc D PC D PC D Exc E E W PC E Exc M PC M M Illegal Opcode Data Addr Except Kill Writeback Overflow Select Handler PC PC Address Exceptions Cause EPC Kill F Stage Kill D Stage Kill E Stage Asynchronous Interrupts Exception Handling(In-Order Five-Stage Pipeline) • Hold exception flags in pipeline until commit point (M stage) • Exceptions in earlier pipe stages override later exceptions • Inject external interrupts at commit point (override others) • If exception at commit: update Cause and EPC registers, kill • all stages, inject handler PC into fetch stage CS252-S11, lecture 7

39. W PC D X2 W Xn X2 Xn X3 X1 X1 X2 X3 X2 X3 X3 Xn Xn Complex In-Order Pipeline: Precise Exceptions • Delay writeback so all operations have same latencyto W stage • Write ports never oversubscribed (one inst. in & one inst. out every cycle) • Instructions commit in order, simplifies precise exception implementation • How to prevent increase latency for single-cycle ops? • Bypassing • However: can be very expensive • Other downside: no out-of-order execution Data Mem Inst. Mem Decode GPRs + FPRs Fadd Commit Point Fmul FDiv Unpipelined divider CS252-S11, lecture 7

40. In-Order Commit for Precise Exceptions In-order Out-of-order In-order Commit Fetch Decode Reorder Buffer Kill Kill Kill Exception? Execute Inject handler PC • Instructions fetched and decoded into instruction • reorder buffer in-order • Execution is out-of-order (  out-of-order completion) • Commit (write-back to architectural state, i.e., regfile & • memory) is in-order Temporary storage needed to hold results before commit (shadow registers and store buffers) CS252-S11, lecture 7

41. Reorder Buffer FP Op Queue Compar network Program Counter Exceptions? Dest Reg FP Regs Result Valid Reorder Table Res Stations Res Stations FP Adder FP Adder What are the hardware complexities with reorder buffer (ROB)? • How do you find the latest version of a register? • As specified by Smith paper, need associative comparison network • Could use future file or just use the register result status buffer to track which specific reorder buffer has received the value • Need as many ports on ROB as register file CS252-S11, lecture 7

42. Four Steps of Speculative Tomasulo 1. Issue—get instruction from FP Op Queue If reservation station and reorder buffer slot free, issue instr & send operands &reorder buffer no. for destination (this stage sometimes called “dispatch”) 2. Execution—operate on operands (EX) When both operands ready then execute; if not ready, watch CDB for result; when both in reservation station, execute; checks RAW (sometimes called “issue”) 3. Write result—finish execution (WB) Write on Common Data Bus to all awaiting FUs & reorder buffer; mark reservation station available. 4. Commit—update register with reorder result When instr. at head of reorder buffer & result present, update register with result (or store to memory) and remove instr from reorder buffer. Mispredicted branch flushes reorder buffer (sometimes called “graduation”) CS252-S11, lecture 7

43. 1 10+R2 Tomasulo With Reorder buffer: Done? FP Op Queue ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1 Newest Reorder Buffer Oldest F0 LD F0,10(R2) N Registers To Memory Dest from Memory Dest Dest Reservation Stations FP adders FP multipliers CS252-S11, lecture 7

44. F10 ADDD F10,F4,F0 N F0 LD F0,10(R2) N 1 10+R2 Tomasulo With Reorder buffer: Done? FP Op Queue ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1 Newest Reorder Buffer Oldest Registers To Memory Dest from Memory Dest 2 ADDD R(F4),ROB1 Dest Reservation Stations FP adders FP multipliers CS252-S11, lecture 7

45. F2 DIVD F2,F10,F6 N F10 ADDD F10,F4,F0 N F0 LD F0,10(R2) N 3 DIVD ROB2,R(F6) 1 10+R2 Tomasulo With Reorder buffer: Done? FP Op Queue ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1 Newest Reorder Buffer Oldest Registers To Memory Dest from Memory Dest 2 ADDD R(F4),ROB1 Dest Reservation Stations FP adders FP multipliers CS252-S11, lecture 7

46. F0 ADDD F0,F4,F6 N F4 LD F4,0(R3) N -- BNE F2,<…> N F2 DIVD F2,F10,F6 N F10 ADDD F10,F4,F0 N F0 LD F0,10(R2) N 3 DIVD ROB2,R(F6) Tomasulo With Reorder buffer: Done? FP Op Queue ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1 Newest Reorder Buffer Oldest Registers To Memory Dest from Memory Dest 2 ADDD R(F4),ROB1 6 ADDD ROB5, R(F6) Dest Reservation Stations 1 10+R2 6 0+R3 FP adders FP multipliers CS252-S11, lecture 7

47. -- ROB5 ST 0(R3),F4 N F0 ADDD F0,F4,F6 N F4 LD F4,0(R3) N -- BNE F2,<…> N F2 DIVD F2,F10,F6 N F10 ADDD F10,F4,F0 N F0 LD F0,10(R2) N 3 DIVD ROB2,R(F6) Tomasulo With Reorder buffer: Done? FP Op Queue ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1 Newest Reorder Buffer Oldest Registers To Memory Dest from Memory Dest 2 ADDD R(F4),ROB1 6 ADDD ROB5, R(F6) Dest Reservation Stations 1 10+R2 6 0+R3 FP adders FP multipliers CS252-S11, lecture 7

48. -- M[10] ST 0(R3),F4 Y F0 ADDD F0,F4,F6 N F4 M[10] LD F4,0(R3) Y -- BNE F2,<…> N F2 DIVD F2,F10,F6 N F10 ADDD F10,F4,F0 N F0 LD F0,10(R2) N 3 DIVD ROB2,R(F6) 1 10+R2 Tomasulo With Reorder buffer: Done? FP Op Queue ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1 Newest Reorder Buffer Oldest Registers To Memory Dest from Memory Dest 2 ADDD R(F4),ROB1 6 ADDD M[10],R(F6) Dest Reservation Stations FP adders FP multipliers CS252-S11, lecture 7

49. -- M[10] ST 0(R3),F4 Y F0 <val2> ADDD F0,F4,F6 Ex F4 M[10] LD F4,0(R3) Y -- BNE F2,<…> N F2 DIVD F2,F10,F6 N F10 ADDD F10,F4,F0 N F0 LD F0,10(R2) N 3 DIVD ROB2,R(F6) 1 10+R2 Tomasulo With Reorder buffer: Done? FP Op Queue ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1 Newest Reorder Buffer Oldest Registers To Memory Dest from Memory Dest 2 ADDD R(F4),ROB1 Dest Reservation Stations FP adders FP multipliers CS252-S11, lecture 7

50. -- M[10] ST 0(R3),F4 Y F0 <val2> ADDD F0,F4,F6 Ex F4 M[10] LD F4,0(R3) Y -- BNE F2,<…> N What about memory hazards??? 3 DIVD ROB2,R(F6) 1 10+R2 Tomasulo With Reorder buffer: Done? FP Op Queue ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1 Newest Reorder Buffer F2 DIVD F2,F10,F6 N F10 ADDD F10,F4,F0 N Oldest F0 LD F0,10(R2) N Registers To Memory Dest from Memory Dest 2 ADDD R(F4),ROB1 Dest Reservation Stations FP adders FP multipliers CS252-S11, lecture 7