Complexity-Effective Issue Queue Design Under Load-Hit Speculation

1. Complexity-Effective Issue Queue Design Under Load-Hit Speculation Tali Moreshet and R. Iris Bahar Brown University Division of Engineering

2. Motivation • Pipelines are getting deeper • Higher clock frequencies • Increased architectural complexity • Speculatively issued instructions are particularly sensitive to pipeline depth • Branch prediction • Load hit prediction WCED 2002

3. Pipeline Load Resolution Loop forwarding Issue Queue Register File Register Rename Unit Instruction Cache Functional Units Data Cache Fetch Decode Issue Execute WCED 2002

4. Load Hit Prediction • Issue instructions dependent on load as soon as possible • Assume load hits in DL1 BUT… • Load hit status is known only after dependent instructions may issue WCED 2002

5. Example Cycle: 1 2 3 4 5 6 7 8 Issue Exec Exec Exec LOAD ADD Issue Exec SUB Issue Exec MULT Issue Exec Speculative window WCED 2002

6. Issue Exec Issue Exec Issue Speculative window Example Cycle: 1 2 3 4 5 6 7 8 9 Issue Exec Exec Exec LOAD ADD SUB MULT Exec WCED 2002

7. Exec Exec Issue Exec Issue Issue Speculative window Example Cycle: 1 2 3 4 5 6 7 8 9 10 Issue Exec Exec LOAD ADD SUB Exec MULT WCED 2002

8. What Happens On a Load Miss? • Re-issue instructions in speculative window after a load miss • Keep post-issue instructions in issue queue long enough to ensure re-issuing will not be necessary WCED 2002

9. Complexity-Effective Load Hit Speculation • As pipeline depth increases: • Retain performance benefit • Consider complexity of re-issue and prediction policies • Consider impact on issue queue design WCED 2002

10. Re-Issue Policies • 4 different load hit speculation policies: • No load hit speculation • Perfect load hit speculation • Replay only instructions dependent on load that missed • Replay all instructions in speculative window • Load hit/miss predictor to limit re-issuing WCED 2002

11. Performance Impact WCED 2002

12. Impact on Issue Queue Occupancy WCED 2002

13. Impact on Issue Queue Occupancy WCED 2002

14. Impact on Issue Queue Occupancy • As pipeline depth increases: • Issue queue gets cluttered with post-issue instructions (average 55%) • Limits the available ILP • Inefficient use of complexity in instruction bid/grant arbitration logic WCED 2002

15. The Bid / Grant Loop Issue Queue req grant req grant N-wide M entries ... Bid for issue slot Broadcast grant req grant Prioritize & Select WCED 2002

16. Issue Queue Utilization Problem • Complexity of bid/grant arbitration logic increases with size of the IQ • IQ consists largely of post-issue instructions • Limiting the available ILP that a large IQ is supposed to provide • Not a complexity-effective design WCED 2002

17. IQ Design Options • Increase the IQ size • Improve performance – increase available ILP • Increase complexity • Simplify arbitration logic – use slower circuitry • Reduce complexity • Hurt performance • Reduce IQ size • Reduce complexity • Hurt performance WCED 2002

18. Double Latency of Issue Queue WCED 2002

19. Smaller IQ (48 Entry) WCED 2002

20. Complexity-Effective Issue Queue • Goal • Reduce complexity • Do not degrade performance • Solution: The Dual Issue Queue • Move post-issue instructions from main queue to separate replay queue • Increase available ILP • Reduce size of main IQ WCED 2002

21. Dual Issue Queue MIQ Main Issue Queue from Fetch unit Register File Register Rename Unit Functional Units Data Cache Replay Issue Queue RIQ Replay_req WCED 2002

22. Dual Issue Queue Performance WCED 2002

23. Conclusion • Load hit speculation is critical for high performance in deeper pipelines • Larger percentage of post-issue instructions in issue queue • Complexity-effective issue queue scheme addresses utilization problem • For deepest pipelines, overall performance improves while reducing complexity of IQ WCED 2002