PaCo Probability-Based Pa th Co nfidence Prediction

1. PaCoProbability-Based Path Confidence Prediction Kshitiz Malik, Mayank Agarwal, Vikram Dhar, Matthew Frank Implicitly Parallel Architectures Group University of Illinois at Urbana-Champaign

2. Summary • Path Confidence: likelihood of correct path • Pipeline Gating, SMT Fetch • Conventional: use count of low-conf branches • Inaccurate • PaCo: Directly estimates goodpath probability • Highly accurate, modest hardware • Improves performance on gating, SMT Fetch HPCA-14 February 18, 2008

3. Outline • Overview • Motivation • Design • Evaluation • Applications HPCA-14 February 18, 2008

4. Branch Confidence Prediction • Branch Confidence: Single Branches • Low Conf / High Conf • Applications: Checkpointing, Multipath etc HPCA-14 February 18, 2008

5. Path Confidence Prediction • Path Confidence: likelihood of being on correct path • Contributions from Multiple Branches • Conventionally: Count of (unexecuted) low-confidence branches. HPCA-14 February 18, 2008

6. Path Confidence: Applications • Path Confidence: Multiple Branches • Count of low-confidence branches. • Applications: Pipeline Gating, SMT Prioritization Gate-Count = 5 (Fetch gated when conf >= 5) Gated HPCA-14 February 18, 2008

7. Path Confidence: Applications • Path Confidence: Multiple Branches • Count of low-confidence branches. • Applications: Pipeline Gating, SMT Prioritization HPCA-14 February 18, 2008

8. Issues with Conventional Approach • Path Confidence: Multiple Branches • Count of low-confidence branches. • Applications: Pipeline Gating, SMT Prioritization • Problem: implicit assumption that • High-conf branches never mispredict • All low-conf branches have same misprediction probability HPCA-14 February 18, 2008

9. Path Confidence using PaCo • Directly estimates goodpath probability • Highly Accurate: RMS error 3.8% • Modest Hardware: 60 bytes of counters • Excellent Performance in applications • Gating: BadpathInstrs. Performance Conv↓ 7%↓ 0.1% PaCo ↓ 32%↓ 0.01% • SMT Fetch Prioritization: ↑ perfupto 23% (5.5% av.) HPCA-14 February 18, 2008

10. Outline • Overview • Motivation • Design • Evaluation • Applications HPCA-14 February 18, 2008

11. Branch Confidence Prediction • Classify branches as low conf. or high conf. • JRS predictor: • Count consecutive correct predictions • Below threshold: Low Confidence • Table of Miss-Distance Counters (MDCs) MDC Value < Threshold? + MDC Table MDC Value 5 Mispredict? 1 Branch PC 0 + On Branch Execution Global Hist HPCA-14 February 18, 2008 4 bits

12. Conventional Path Confidence Prediction • Count of low-confidence branches = measure of path confidence • Threshold-and-Count Approach • Inaccurate • Coarseness • No relation to goodpath probability MDC Table Threshold Function Sum Low Conf / High Conf (1 bit) Miss Distance Counter Value (4 bits) Path Confidence Branch HPCA-14 February 18, 2008

13. Threshold Function Low Conf / High Conf (1 bit) Coarseness Miss Distance Counter Value (4 bits) a JRS Threshold = 3 • All low-conf branches are equal • Eg: SMT prior. • gcc: • a pending • 0.57 gpathprob • vortex: • 2 bpend. • Prob = 0.88*0.88 = .78 • yet, fetch from gcc! b High Conf Low Conf HPCA-14 February 18, 2008

14. Threshold Function Low Conf / High Conf (1 bit) Coarseness Miss Distance Counter Value (4 bits) • All low-conf branches are equal • High-conf branches don’t mispredict • twolf, vortex, mdc=3 • Don’t affect conf. HPCA-14 February 18, 2008

15. Sum ≠ Probability Sum Low Conf / High Conf (1 bit) Path Confidence • Gating at count=5 • Too aggressive for gzip, not useful for route • SMT Fetch: • gzip and route, conf = 5. Equal bandwith? Goodpath Likelihood when 5 low-confidence branches are pending HPCA-14 February 18, 2008

16. Sum ≠ Probability Goodpath Likelihood when 5 low-confidence branches are pending HPCA-14 February 18, 2008

17. Sum ≠ Probability Sum Low Conf / High Conf (1 bit) Path Confidence • Hard to choose optimal gate-count • Different gate-counts for different benchmarks • Different gate-counts for different phases • Hard to compare path confidence of diff. apps • SMT Fetch Prioritization sub-optimal HPCA-14 February 18, 2008

18. Outline • Overview • Motivation • Design • Evaluation • Applications HPCA-14 February 18, 2008

19. Design • Calculate Goodpath Probability directly • Finding ‘correct prediction probability’ for a branch • MDC table good differentiator of misprediction rates • Find misprediction rate for each MDC value • No thresholding! • Other, more h/w intensive approaches possible HPCA-14 February 18, 2008

20. Threshold-and-count vs. PaCo MDC Table Threshold Function Sum Low Conf / High Conf (1 bit) Miss Distance Counter Value (4 bits) Path Confidence Branch Mispredict Rate Calculator (MRT) Product Mispredict Probability Path Confidence Mispred Rate Table 0 PaCo 1 2 . . . Misprediction Rate 13 14 15 HPCA-Practice Talk February 13, 2008 MDC Value

21. Hardware Complexity • Remove floating point: scale to integer values • Hardware Complexity • Floating point MUL and DIV required • Use logarithms, remove mul/div HPCA-14 February 18, 2008

22. MDC Table PaCo Hardware 5 Branch PC Feedback from Backend 60 bytes of counters, 10-bit shift register Global Hist Mispredict Rate Calculator Path Confidence Predictor Encoded Probability Correct Preds MDC 0 Mispreds MDC 0 MDC 1 Correct Preds MDC 1 Log Circuit Mispreds . . . . . . . . . . . Path Confidence 6 bits + + 10 bits Correct Preds MDC 15 MDC 15 Mispreds 12 bits

23. Outline • Overview • Motivation • Design • Evaluation • Applications HPCA-14 February 18, 2008

24. Evaluation: Prediction Accuracy • RMS Error = 0.038. • Example: • Predicted 60% goodpath likelihood • Should be within (60 ± 3.8) = 56.2% - 63.8% HPCA-14 February 18, 2008

25. Accuracy: Reliability Diagram RMS Error

26. Outline • Overview • Motivation • Design • Evaluation • Applications HPCA-14 February 18, 2008

27. Applications: Pipeline Gating GateCount=1 GateCount=2 GateCount=10 HPCA-14 February 18, 2008

28. Applications: Pipeline Gating 0.1% perf loss 32% redn. in badpath instructions HPCA-14 February 18, 2008

29. Applications: SMT Fetch Prioritization

30. Conclusions • Threshold-and-Count predictors are inaccurate • PaCo: Directly produces goodpath probability • Uses modest h/w by using logarithms • Highly accurate: low RMS error (3.8%) • PaCo does very well in Pipeline Gating and SMT Fetch Prioritization HPCA-14 February 18, 2008

31. Questions? HPCA-14 February 18, 2008

32. Backup1: Comparison with WPUP • WPUP: Perfect Fetch gating improves average performance by 2.3% (excl. mcf and parser) • Badpath Instructions • Good: prefetching (useful: small ROBs, wide machines) • Bad: BTB/Cache pollution (prob: smaller BTBs/caches) • Prefetching much less useful with 512 ROB

33. Pipeline Parameters: Fetch Gating HPCA-14 February 18, 2008

34. Pipeline Parameters: SMT HPCA-14 February 18, 2008

35. MDC Table Threshold Function Sum Low Conf / High Conf (1 bit) Miss Distance Counter Value (4 bits) Path Confidence Branch HPCA-Practice Talk February 13, 2008

36. + MDC Table 5 Mispredict? 1 Branch PC 0 + Global Hist HPCA-14 February 18, 2008 HPCA-14 February 18, 2008 4 bits

37. Applications: SMT Fetch Prioritization