1 / 34

Hardware-Based Speculation

Hardware-Based Speculation. As more instruction-level parallelism is exploited, maintaining control dependences becomes an increasing burden. => Speculating on the outcome of branches and executing the program as if the guesses were correct. Hardware Speculation.

ugo
Download Presentation

Hardware-Based Speculation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Hardware-Based Speculation • As more instruction-level parallelism is exploited, maintaining control dependences becomes an increasing burden. => Speculating on the outcome of branches and executing the program as if the guesses were correct. • Hardware Speculation CSCE 614 Fall 2009

  2. 3 Key Ideas of Hardware Speculation • Dynamic Branch Prediction • Choose which instruction to execute. • Speculation • Allow the execution of instructions before the control dependences are resolved (with the ability to undo the effect of an incorrectly speculated sequence). • Dynamic Scheduling • Deal with the scheduling of different combinations of basic blocks CSCE 614 Fall 2009

  3. Examples • PowerPC 603/604/G3/G4 • MIPS R10000/12000 • Intel Pentium II/III/4 • Alpha 21264 • AMD K5/K6/Athlon CSCE 614 Fall 2009

  4. Hardware Speculation • Extended Tomasulo’s algorithm • Additional step (instruction commit) required • Allow instructions to execute out-of-order but to force them to commit in order. • Any irrevocable action (updating state or taking an exception) is prevented until an instruction commits. CSCE 614 Fall 2009

  5. Reorder Buffer (ROB) • Holds the result of an instruction between the time the operation associated with the instruction completes and the time the instruction commits. • Source of operands for instructions • With speculation, the register file (or memory) is not updated until the instruction commits. CSCE 614 Fall 2009

  6. ROB Fields • Instruction type: indicates whether the instruction is a branch, a store, or a register operation (ALU or Load). • Destination: supplies the register number (for loads and ALU operations) or the memory address (for stores). • Value: holds the value of the instruction result until the instruction commits. • Ready: indicates that the instruction has completed execution, and the value is ready. CSCE 614 Fall 2009

  7. Hardware Speculation Commit (write to RF, MEM) Execute Write result (to ROB) Issue Reorder Buffer (ROB) Reservation Stations CSCE 614 Fall 2009

  8. Basic Structure of MIPS FP Unit The ROB completely replaces the store buffer. The renaming function of the reservation stations is replaced by the ROB CSCE 614 Fall 2009

  9. 4 Steps of Execution • Issue (also called “dispatch”) - Get an instruction from the instruction queue. - Issue the instruction if there is an empty reservation station and an empty slot in ROB. - If either all reservation stations are full or the ROB is full, then instruction issue is stalled. CSCE 614 Fall 2009

  10. 2. Execute - If one or more of the operands is not yet available, monitor the CDB while waiting for the register to be computed. - Also RAW hazards are checked. - When both operands are available at a reservation station, execute the operation. - Loads require two steps (effective address calculation and source operand). - Stores need one step (effective address calculation). CSCE 614 Fall 2009

  11. 3. Write Result - When the result is available, write it on the CDB and from the CDB into the ROB, as well as to any reservation stations waiting for this result. - For stores, if the value to be stored is available, it is written into the Value field of the ROB entry for the store. CSCE 614 Fall 2009

  12. 4. Commit (also called “completion” or “graduation”) - Normal commit: When an instruction reaches the head of the ROB and its result is present in the buffer, the processor updates the register with the result and removes the instruction from the ROB. - Store commit: Similar except that memory is updated. - Branch with an incorrect prediction: The speculation is wrong. The ROB is flushed and execution is restarted at the correct successor of the branch. CSCE 614 Fall 2009

  13. Example L.D F6, 34(R2) L.D F2, 45(R3) MUL.D F0, F2, F4 SUB.D F8, F6, F2 DIV.D F10, F0, F6 ADD.D F6, F8, F2 When the MUL.D is ready to commit. CSCE 614 Fall 2009

  14. Example (p.109) Loop: L.D F0, 0(R1) MUL.D F4, F0, F2 S.D F4, 0(R1) DADDIU R1, R1, #-8 BNE R1, R2, Loop Assume that we have issued all the instructions in the loop twice. Assume that L.D and MUL.D from the first iteration have committed and all other instructions have completed execution. Show the contents of the ROB and the FP registers. CSCE 614 Fall 2009

  15. Hardware Speculation • Because neither the register values nor any memory values are actually written until an instruction commits, the processor can easily undo its speculative actions when a branch is found to be mispredicted. • Exceptions are handled by not recognizing the exception until it is ready to commit. CSCE 614 Fall 2009

  16. Hardware Speculation • Figure 2.17 (p.113) CSCE 614 Fall 2009

  17. Multiple-Issue Processors • Allow multiple instructions to issue in a clock cycle. • Ideal CPI < 1 • 3 flavors • Statically Scheduled Superscalar • Dynamically Scheduled Superscalar • VLIW (Very Long Instruction Word) CSCE 614 Fall 2009

  18. Superscalar Processors • Issue varying numbers of instructions per clock • statically scheduled • using compiler techniques • in-order execution • dynamically scheduled • Tomasulo’s algorithm • out-of-order execution CSCE 614 Fall 2009

  19. VLIW Processors • issue a fixed number of instructions formatted either as one large instruction or as a fixed instruction packet with the parallelism among instructions explicitly indicated by the instruction (EPIC: Explicitly Parallel Instruction Computers). • Statically scheduled by the compiler. CSCE 614 Fall 2009

  20. CSCE 614 Fall 2009

  21. Multiple Instruction Issue with Dynamic Scheduling • Two-issue dynamically scheduled processor • It can issue any pair of instructions if there are reservation stations of the right type available. • Extended Tomasulo’s algorithm Note that Tomasulo’s algorithm (and Hardware Speculation) is used for both integer operations and FP operations. CSCE 614 Fall 2009

  22. Two approaches to implement • Issue one instruction in half a clock cycle, so that two instructions can be processed in one clock cycle. • Build the logic necessary to handle two instructions at once, including any possible dependences between the instructions. • Modern superscalar processors that issue 4 or more instructions per clock often include both approaches. CSCE 614 Fall 2009

  23. How to Handle Branches? • Dynamically scheduled processors • Only allow instructions to be fetched and issued (but not actually executed) until the branch has completed. • IBM 360/91 • Processors with hardware speculation • Can actually execute instructions based on branch prediction. CSCE 614 Fall 2009

  24. Note that we consider loads and stores, including those to FP registers, as integer operations. • Assume that FP adds take 3 execution cycles. • Latency: Write CDB Execute CSCE 614 Fall 2009

  25. The throughput improvement versus a single-issue pipeline is small. • There is only one FP operation per iteration. • There is only one Integer ALU for both integer ALU operations and effective address calculations. • Larger improvements would be possible if the processor could execute more integer operations per cycle. CSCE 614 Fall 2009

  26. Multiple Issue with Speculation • We process multiple instructions per clock assigning reservation stations and reorder buffers to the instructions. • To maintain throughput of greater than one instruction per cycle, a speculative processor must be able to handle multiple instruction commits per clock cycle. CSCE 614 Fall 2009

  27. Example (p.119) Loop: LD R2, 0(R1) DADDIU R2, R2, #1 SD R2, 0(R1) DADDIU R1, R1, #8 BNE R2, R3, Loop Consider the execution of the loop on a two-issue processor, once without speculation (dynamic scheduling/Tomasulo’s algorithm) and once with speculation. Assume that there are separate integer functional units for effective address calculation, for ALU operations, and for branch condition evaluation. Assume that there are 2 CDBs. Assume that up to two instructions of any type can commit per clock for a processor with speculation. Show the execution timing of the first three iterations of the loop. CSCE 614 Fall 2009

  28. High-Performance Instruction Delivery • For multiple-issue (delivering 4~8 instructions per clock cycle) processors • Branch-target buffers • Integrated instruction fetch unit • Return address prediction CSCE 614 Fall 2009

  29. Branch-Target Buffers • To reduce the branch penalty for the classic 5-stage pipeline, we want to know what address to fetch by the end of IF. • Branch-target buffer: a branch-prediction cache that stores the predicted address for the next instruction after a branch. • We access the buffer during the IF stage using the instruction address. (We don’t know what the instruction is.) CSCE 614 Fall 2009

  30. Branch-Target Buffers Optional. May be used for extra prediction state bits. Branch-Target Cache CSCE 614 Fall 2009

  31. Branch-Target Buffers • We only need to store the predicted-taken branches in the branch-target buffer. • Why? • No branch delay if a branch-prediction entry is found and is correct. CSCE 614 Fall 2009

  32. CSCE 614 Fall 2009

  33. Return Address Predictors • Predicting indirect jumps (destination address varies at run time) • Procedure returns, procedure calls, case, select, etc. • SPEC89: 85% of indirect jumps are procedure returns. • A small buffer of return addresses operating as a stack • Caches the most recent return addresses • Push a return address on the stack at a call • Pop one off at a return CSCE 614 Fall 2009

  34. Integrated Instruction Fetch Units • A separate autonomous unit that feeds instructions to the rest of the pipeline for multiple-issue processors. • Have several functions • Integrated branch prediction • Instruction prefetch • Instruction memory access and buffering CSCE 614 Fall 2009

More Related