Hyper-Threading Technology Architecture and Micro-Architecture

1. Hyper-Threading Technology Architecture and Micro-Architecture Prepared by Tahir Celebi Istanbul, 2005 Tahir CELEBI, Istanbul, 2005

2. Outline • Introduction • Traditional Approaches • Hyper-Threading Overview • Hyper-Threading Implementation • Front-End Execution • Out-of-Order Execution • Performance Results • OS Supports • Conclusion Tahir CELEBI, Istanbul, 2005

3. Introduction • Hyper-Threading technology makes a single processor appear as two logical processors. • It was first implemented in the Prestonia version of the Pentium® 4 Xeon processor on 02/25/02. Tahir CELEBI, Istanbul, 2005

4. Traditional Approaches (I) • High requirements of Internet and Telecommunications Industries • Results are unsatisfactory compared the gain they provide with the cost they cause • Well-known techniques; • Super Pipelining • Branch Prediction • Super-scalar Execution • Out-of-order Execution • Fast memories (Caches) Tahir CELEBI, Istanbul, 2005

5. Traditional Approaches (II) • Super Pipelining: • Have finer granularities, execute far more instructions within a second (Higher clock frequencies) • Hard to handle cache misses, interrupts and branch mispredictions • Instruction Level Parallelism (ILP) • Mainly targets to increase the number of instructions within a cycle • Super Scalar Processors with multiple parallel execution units • Execution needs to be verified for out-of-order execution • Fast Memory (Caches) • To reduce the memory latencies, hierarchical units are using which are not an exact solution Tahir CELEBI, Istanbul, 2005

6. Traditional Approaches (III) Same silicon technology Normalized speed-ups with Intel486™ Tahir CELEBI, Istanbul, 2005

7. Thread-Level Parallelism • Chip Multi-Processing (CMP) • Put 2 processors on a single die • Processors (only) may share on-chip cache • Cost is still high • IBM Power4 PowerPC chip • Single Processor Multi-Threading; • Time-sliced multi-threading • Switch-on-event multi-threading • Simultaneous multi-threading Tahir CELEBI, Istanbul, 2005

8. Hyper-Threading (HT) Technology • Provides more satisfactory solution • Single physical processor is shared as two logical processors • Each logical processor has its own architecture state • Single set of execution units are shared between logical processors • N-logical PUs are supported • Have the same gain % with only 5% die-size penalty. • HT allows single processor to fetch and execute two separate code streams simultaneously. Tahir CELEBI, Istanbul, 2005

9. HT Resource Types • Replicated Resources • Flags, Registers, Time-Stamp Counter, APIC • Shared Resources • Memory, Range Registers, Data Bus • Shared | Partitioned Resources • Caches & Queues Tahir CELEBI, Istanbul, 2005

10. HT Pipeline (I) Tahir CELEBI, Istanbul, 2005

11. HT Pipeline (II) Tahir CELEBI, Istanbul, 2005

12. HT Pipeline (III) Tahir CELEBI, Istanbul, 2005

13. Execution Trace Cache (TC) (I) • Stores decoded instructions called “micro-operations” or “uops” • Arbitrate access to the TC using two IPs • If both PUs ask for access then switch will occur in the next cycle. • Otherwise, access will be taken by the available PU • Stalls (stem from misses) lead to switch • Entries are tagged with the owner thread info • 8-way set associative, Least Recently Used (LRU) algorithm • Unbalanced usage between processors Tahir CELEBI, Istanbul, 2005

14. Execution Trace Cache (TC) (I) Tahir CELEBI, Istanbul, 2005

15. Microcode Store ROM (MSROM) (I) • Complex instructions (e.g. IA-32) are decoded into more than 4 uops • Invoked by Trace Cache • Shared by the logical processors • Independent flow for each processor • Access to MSROM alternates between logical processors as in the TC Tahir CELEBI, Istanbul, 2005

16. Microcode Store ROM (MSROM) (II) Tahir CELEBI, Istanbul, 2005

17. ITLB and Branch Prediction (I) • If there is a TC miss, bytes need to be loaded from L2 cache and decoded into TC • ITLB gets the “instruction deliver” request • ITLB translates next Pointer address to the physical address • ITLBs are duplicated for processors • L2 cache arbitrates on first-come first-served basis while always reserve at least one slot for each processor • Branch prediction structures are either duplicated or shared • If shared owner tags should be included Tahir CELEBI, Istanbul, 2005

18. ITLB and Branch Prediction (II) Tahir CELEBI, Istanbul, 2005

19. Uop Queue Tahir CELEBI, Istanbul, 2005

20. HT Pipeline (III) -- Revisited Tahir CELEBI, Istanbul, 2005

21. Allocator • Allocates many of the key machine buffers; • 126 re-order buffer entries • 128 integer and floating-point registers • 48 load, 24 store buffer entries • Resources shared equal between processors • Limitation of the key resource usage, we enforce fairness and prevent deadlocks over the Arch. • For every clock cycle, allocator switches between uop queues • If there is stall or HALT, there is no need to alternate between processors Tahir CELEBI, Istanbul, 2005

22. Register Rename • Involves with mapping shared registers names for each processor • Each processor has its own Register Alias Table (RAT) • Uops are stored in two different queues; • Memory Instruction Queue (Load/Store) • General Instruction Queue (Rest) • Queues are partitioned among PUs Tahir CELEBI, Istanbul, 2005

23. Instruction Scheduling • Schedulers are at the heart of the out-of-order execution engine • There are five schedulers which have queues of size 8-12 • Scheduler is oblivious when getting and dispatching uops • It ignores the owner of the uops • It only considers if input is ready or not • It can get uops from different PUs at the same time • To provide fairness and prevent deadlock, some entries are always assigned to specific PUs Tahir CELEBI, Istanbul, 2005

24. Execution Units & Retirement • Execution Units are oblivious when getting and executing uops • Since resource and destination registers were renamed earlier, during/after the execution it is enough to access physical registries • After execution, the uops are placed in the re-order buffer which decouples the execution stage from retirement stage • The re-order buffer is partitioned between PUs • Uop retirement commits the architecture state in program order • Once stores have retired, the store data needs to be written into L1 data-cache, immediately Tahir CELEBI, Istanbul, 2005

25. Memory Subsystem • Totally oblivious to logical processors • Schedulers can send load or store uops without regard to PUs and memory subsystem handles them as they come • Memory types; • DTLB: • Translates addresses to physical addresses • 64 fully associative entries; each entry can map either 4K or 4MB page • Shared between PUs (Tagged with ID) • L1, L2 and L3 caches • Cache conflict might degrade performance • Using same data might increase performance (more mem. hits) Tahir CELEBI, Istanbul, 2005

26. System Modes • Two modes of operation; • single-task (ST) • When there is one SW thread to execute • multi-task (MT) • When there are more than one SW threads to execute • ST0 or ST1 where number shows the active PU • HALT command was introduced where resources are combined after the call • Reason is to have better utilization of resources Tahir CELEBI, Istanbul, 2005

27. Performance Tahir CELEBI, Istanbul, 2005

28. OS Support for HT • Native HT Support • Windows XP Pro Edition • Windows XP Home Edition • Linux v 2.4.x (and higher) • Compatible with HT • Windows 2000 (all versions) • Windows NT 4.0 (limited driver support) • No HT Support • Windows ME • Windows 98 (and previous versions) Tahir CELEBI, Istanbul, 2005

29. Measured performance (Xeon) showed performance gains of up to 30% on common server applications. HT is expected to be viable and market standard from Mobile to server processes. Conclusion Tahir CELEBI, Istanbul, 2005

30. Questions ? Tahir CELEBI, Istanbul, 2005