1 / 39

CSCE 432/832 High Performance Processor Architectures Instruction Flow & Branch Prediction

CSCE 432/832 High Performance Processor Architectures Instruction Flow & Branch Prediction. Adopted from Lecture notes based in part on slides created by Mikko H. Lipasti, John Shen, Mark Hill, David Wood, Guri Sohi, and Jim Smith. Instruction Flow Techniques.

juliahoward
Download Presentation

CSCE 432/832 High Performance Processor Architectures Instruction Flow & Branch Prediction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CSCE 432/832 High Performance Processor ArchitecturesInstruction Flow & Branch Prediction Adopted from Lecture notes based in part on slides created by Mikko H. Lipasti, John Shen, Mark Hill, David Wood, Guri Sohi, and Jim Smith

  2. Instruction Flow Techniques • Goal of Instruction Flow and Impediments • Branch Types and Implementations • What’s So Bad About Branches? • What are Control Dependences? • Impact of Control Dependences on Performance • Improving I-Cache Performance CSCE 432/832, Superscalar -- Instruction Flow

  3. Instruction Flow in Context CSCE 432/832, Superscalar -- Instruction Flow

  4. Goal and Impediments • Goal of Instruction Flow • Supply processor with maximum number of useful instructions every clock cycle • Impediments • Branches and jumps • Finite I-Cache CSCE 432/832, Superscalar -- Instruction Flow

  5. Branch Types and Implementation • Types of Branches • Conditional or Unconditional • Save PC? • How is target computed? • Single (fixed-range) target (immediate, PC+immediate) • Multiple (varied and flexible) targets (register) • Branch Architectures • Condition code or condition registers • Register CSCE 432/832, Superscalar -- Instruction Flow

  6. Branch Types and Implementation • PowerPC 32-bit condition register – eight 4-bit fields (CR0-CR7) CR0 can be implicit result of integer op CR1 can be implicit result of FP op Compare ops set CR fields Special CR ops manipulate bits Conditional branch instructions test CR bits 31 0 CR0 CR1 CR2 CR3 CR4 CR5 CR6 CR7 CSCE 432/832, Superscalar -- Instruction Flow

  7. Branches – PowerPC 12 Types of Branches Branch (unconditional, no save PC, PC+imm) Branch absolute (uncond, no save PC, imm) Branch and link (uncond, save PC, PC+imm) Branch abs and link (uncond, save PC, imm) Branch conditional (conditional, no save PC, PC+imm) Branch cond abs (cond, no save PC, imm) Branch cond and link (cond, save PC, PC+imm) Branch cond abs and link (cond, save PC, imm) Branch cond to link register (cond, don’t save PC, reg) Branch cond to link reg and link (cond, save PC, reg) Branch cond to count reg (cond, don’t save PC, reg) Branch cond to count reg and link (cond, save PC, reg) CSCE 432/832, Superscalar -- Instruction Flow

  8. Branches – Alpha • Alpha 3 Types of Branches Conditional branch (cond, no save PC, PC+imm) Bxx Ra, disp Unconditional branch (uncond, Save PC, PC+imm) Br Ra, disp Jumps (uncond, save PC, Register) J Ra CSCE 432/832, Superscalar -- Instruction Flow

  9. Branches – MIPS • MIPS 6 Types of Branches Jump (uncond, no save PC, imm) Jump and link (uncond, save PC, imm) Jump register (uncond, no save PC, register) Jump and link register (uncond, save PC, register) Branch (conditional, no save PC, PC+imm) Branch and link (conditional, save PC, PC+imm) CSCE 432/832, Superscalar -- Instruction Flow

  10. What’s So Bad About Branches? • Effects of Branches • Fragmentation of I-Cache lines • Need to determine branch direction • Need to determine branch target • Use up execution resources CSCE 432/832, Superscalar -- Instruction Flow

  11. What’s So Bad About Branches? Problem: Fetch stalls until direction is determined Solutions: • Minimize delay • Move instructions determining branch condition away from branch • Make use of delay • Non-speculative: • Fill delay slots with useful safe instructions • Execute both paths (eager execution) • Speculative: • Predict branch direction CSCE 432/832, Superscalar -- Instruction Flow

  12. What’s So Bad About Branches? Problem: Fetch stalls until branch target is determined Solutions: • Minimize delay • Generate branch target early • Make use of delay: Predict branch target • Single target • Multiple targets CSCE 432/832, Superscalar -- Instruction Flow

  13. Control Dependences • Control Flow Graph • Shows possible paths of control flow through basic blocks CSCE 432/832, Superscalar -- Instruction Flow

  14. Control Dependences • Control Dependence • Node B is CD on Node A if A determines whether B executes CSCE 432/832, Superscalar -- Instruction Flow

  15. Limits on Instruction Level Parallelism (ILP) CSCE 432/832, Superscalar -- Instruction Flow

  16. Riseman and Foster’s Study • 7 benchmark programs on CDC-3600 • Assume infinite machines • Infinite memory and instruction stack • Infinite register file • Infinite functional units • True dependencies only at dataflow limit • If bounded to single basic block, speedup is 1.72 (Flynn’s bottleneck) • If one can bypass branches (hypothetically), then: CSCE 432/832, Superscalar -- Instruction Flow

  17. Improving I-Cache Performance • Larger Cache Size • More associativity • Larger line size • Prefetching • Next-line • Target • Markov • Code layout • Other types of cache organization • Trace cache [Rotenberg paper on reading list] CSCE 432/832, Superscalar -- Instruction Flow

  18. Lecture Overview • Program control flow • Implicit sequential control flow • Disruptions of sequential control flow • Branch Prediction • Branch instruction processing • Branch instruction speculation • Key historical studies on branch prediction • UCB Study [Lee and Smith, 1984] • IBM Study [Nair, 1992] • Branch prediction implementation (PPC 604) • BTAC and BHT design • Fetch Address Generation CSCE 432/832, Superscalar -- Instruction Flow

  19. Program Control Flow • Implicit Sequential Control Flow • Static Program Representation • Control Flow Graph (CFG) • Nodes = basic blocks • Edges = Control flow transfers • Physical Program Layout • Mapping of CFG to linear program memory • Implied sequential control flow • Dynamic Program Execution • Traversal of the CFG nodes and edges (e.g. loops) • Traversal dictated by branch conditions • Dynamic Control Flow • Deviates from sequential control flow • Disrupts sequential fetching • Can stall IF stage and reduce I-fetch bandwidth CSCE 432/832, Superscalar -- Instruction Flow

  20. Program Control Flow • Dynamic traversal of static CFG • Mapping CFG to linear memory CSCE 432/832, Superscalar -- Instruction Flow

  21. Disruption of Sequential Control Flow CSCE 432/832, Superscalar -- Instruction Flow

  22. Branch Prediction • Target address generation Target Speculation • Access register: • PC, General purpose register, Link register • Perform calculation: • +/- offset, autoincrement, autodecrement • Condition resolution Condition speculation • Access register: • Condition code register, General purpose register • Perform calculation: • Comparison of data register(s) CSCE 432/832, Superscalar -- Instruction Flow

  23. Target Address Generation

  24. Condition Resolution

  25. Branch Instruction Speculation CSCE 432/832, Superscalar -- Instruction Flow

  26. Branch/Jump Target Prediction • Branch Target Buffer: small cache in fetch stage • Previously executed branches, address, taken history, target(s) • Fetch stage compares current FA against BTB • If match, use prediction • If predict taken, use BTB target • When branch executes, BTB is updated • Optimization: • Size of BTB: increases hit rate • Prediction algorithm: increase accuracy of prediction CSCE 432/832, Superscalar -- Instruction Flow

  27. Branch Prediction: Condition Speculation • Biased Not Taken • Hardware prediction • Does not affect ISA • Not effective for loops • Software Prediction • Extra bit in each branch instruction • Set to 0 for not taken • Set to 1 for taken • Bit set by compiler or user; can use profiling • Static prediction, same behavior every time • Prediction based on branch offset • Positive offset: predict not taken • Negative offset: predict taken • Prediction based on dynamic history CSCE 432/832, Superscalar -- Instruction Flow

  28. UCB Study [Lee and Smith, 1984] • Benchmarks used • 26 programs (IBM 370, DEC PDP-11, CDC 6400) • 6 workloads (4 IBM, 1 DEC, 1 CDC) • Used trace-driven simulation • Branch types • Unconditional: always taken or always not taken • Subroutine call: always taken • Loop control: usually taken • Decision: either way, if-then-else • Computed goto: always taken, with changing target • Supervisor call: always taken • Execute: always taken (IBM 370) CSCE 432/832, Superscalar -- Instruction Flow

  29. Branch Prediction Function • Prediction function F(X1, X2, … ) • X1 – opcode type • X2 – history • Prediction effectiveness based on opcode only, or history CSCE 432/832, Superscalar -- Instruction Flow

  30. Example Prediction Algorithm • Hardware table remembers last 2 branch outcomes • History of past several branches encoded by FSM • Current state used to generate prediction • Results: CSCE 432/832, Superscalar -- Instruction Flow

  31. Other Prediction Algorithms • Combining prediction accuracy with BTB hit rate (86.5% for 128 sets of 4 entries each), branch prediction can provide the net prediction accuracy of approximately 80%. This implies a 5-20% performance enhancement. CSCE 432/832, Superscalar -- Instruction Flow

  32. IBM Study [Nair, 1992] • Branch processing on the IBM RS/6000 • Separate branch functional unit • Five different branch types • b: unconditional branch • bl: branch and link (subroutine calls) • bc: conditional branch • bcr: condor branch using link register (returns) • bcc: conditional branch using count register • Overlap of branch instructions with other instructions • Zero cycle branches • Two causes for branch stalls • Unresolved conditions • Branches downstream too close to unresolved branches CSCE 432/832, Superscalar -- Instruction Flow

  33. Branch Instruction Distribution CSCE 432/832, Superscalar -- Instruction Flow

  34. Exhaustive Search for Optimal 2-bit Predictor • There are 220 possible state machines of 2-bit predictors • Some machines are uninteresting, pruning them out reduces the number of state machines to 5248 • For each benchmark, determine prediction accuracy for all the predictor state machines • Find optimal 2-bit predictor for each application

  35. Exhaustive Search for Optimal 2-bit Predictor • There are 220 possible state machines of 2-bit predictors • Some machines are uninteresting, pruning them out reduces the number of state machines to 5248 • For each benchmark, determine prediction accuracy for all the predictor state machines • Find optimal 2-bit predictor for each application

  36. Number of History Bits Needed • Branch history table size: Direct-mapped array of 2k entries • Some programs, like gcc, have over 7000 conditional branches • In collisions, multiple branches share the same predictor • Constructive interference • Destructive interference • Marginal gains beyond 1K entries (for these programs) CSCE 432/832, Superscalar -- Instruction Flow

  37. Branch Prediction Implementation (PPC 604) CSCE 432/832, Superscalar -- Instruction Flow

  38. BTAC and BHT Design (PPC 604) CSCE 432/832, Superscalar -- Instruction Flow

  39. BTAC and BHT Design (PPC 604) CSCE 432/832, Superscalar -- Instruction Flow

More Related