1 / 54

Computer Structure The Uncore

Computer Structure The Uncore. 2 nd Generation Intel ® Core ™. Integrates CPU, Graphics, MC , PCI Express* on s ingle c hip . Next Generation Intel ® Turbo Boost Technology. High BW / low-latency core/GFX interconnect. PCH. Substantial performance improvement. × 16 PCIe.

gaius
Download Presentation

Computer Structure The Uncore

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computer StructureThe Uncore

  2. 2ndGeneration Intel® Core™ Integrates CPU, Graphics, MC, PCI Express* on single chip Next Generation Intel® Turbo Boost Technology High BW/ low-latency core/GFX interconnect PCH Substantial performance improvement ×16 PCIe High Bandwidth Last Level Cache PCI Express* DMI System Agent IMC Intel® Advanced Vector Extension (Intel® AVX) Next Generation Graphics and Media Display 2ch DDR3 Core LLC Integrated Memory Controller – 2ch DDR3 Core LLC Embedded DisplayPort Core LLC LLC Core Intel® Hyper-Threading Technology 4 Cores / 8 Threads 2 Cores / 4 Threads Discrete Graphics Support: 1x16 or 2x8 Graphics Foil taken from IDF 2011

  3. 3rdGeneration Intel CoreTM • 22nm process • Quad core die, with Intel HD Graphics 4000 • 1.4 Billion transistors • Die size: 160 mm2

  4. The UncoreSubsystem • The SoCdesign provides a high bandwidth bi-directional ring bus • Connect between the IA cores and the various un-core sub-systems • The uncore subsystem includes • A system agent • The graphics unit (GT) • The last level cache (LLC) • In Intel Xeon Processor E5 Family • No graphics unit (GT) • Instead it contains many more components: • An LLC with larger capacity and snooping capabilities to support multiple processors • Intel® QuickPath Interconnect interfaces that can support multi-socket platforms • Power management control hardware • A system agent capable of supporting highbandwidth traffic from memory and I/O devices PCI Express* DMI System Agent IMC Display Core LLC Core LLC Core LLC LLC Core Graphics From the Optimization Manual

  5. Scalable Ring On-die Interconnect • Ring-based interconnect between Cores, Graphics, Last Level Cache (LLC) and System Agent domain • Composed of 4 rings • 32 Byte Data ring, Request ring, Acknowledgering and Snoop ring • Fully pipelined at core frequency/voltage: bandwidth, latency and power scale with cores • Massive ring wire routing runs over the LLC with no area impact • Access on ring always picks the shortest path – minimize latency • Distributed arbitration, ring protocol handles coherency, ordering, and core interface • Scalable to servers with large number of processors PCI Express* DMI System Agent IMC Display Core LLC Core LLC Core LLC LLC Core Graphics High Bandwidth, Low Latency, Modular Foil taken from IDF 2011

  6. Last Level Cache – LLC • The LLC consists of multiple cache slices • The number of slices is equal to the number of IA cores • Each slice contains a full cache port that can supply 32 bytes/cycle • Each slice has logic portion + data array portion • The logic portion handles • Data coherency • Memory ordering • Access to the data array portion • LLC misses and write-back to memory • The data array portion stores cache lines • May have 4/8/12/16 ways • Corresponding to 0.5M/1M/1.5M/2M block size • The GT sits on the same ring interconnect • Uses the LLC for its data operations as well • May in some case competes with the core on LLC PCI Express* DMI System Agent IMC Display Core LLC Core LLC Core LLC LLC Core Graphics From the Optimization Manual

  7. Cache Box • Interface block • Between Core/Graphics/Media and the Ring • Between Cache controller and the Ring • Implements the ring logic, arbitration, cache controller • Communicates with System Agent for LLC misses, external snoops, non-cacheable accesses • Full cache pipeline in each cache box PCI Express* DMI • Physical Addresses are hashed at the source to prevent hot spots and increase bandwidth • Maintains coherency and ordering for the addresses that are mapped to it • LLC is fully inclusive with “Core Valid Bits” – eliminates unnecessary snoops to cores • Per core CVB indicates if core needs to be snooped for a given cache line • Runs at core voltage/frequency, scales with Cores System Agent IMC Display Core LLC Core LLC Core LLC LLC Core Graphics Distributed coherency & ordering; Scalable Bandwidth, Latency & Power Foil taken from IDF 2011

  8. Ring Interconnect and LLC • The physical addresses of data kept in the LLC are distributed among the cache slices by a hash function • Addresses are uniformly distributed • From the cores and the GT view, the LLC acts as one shared cache • With multiple ports and bandwidth that scales with the number of cores • The number of cache-slices increases with the number of cores • The ring and LLC are not likely to be a BW limiter to core operation • From SW point of view, this does not appear as a normal N-way cache • The LLC hit latency, ranging between 26-31 cycles, depends on • The core location relative to the LLC block(how far the request needs to travel on the ring) • All the traffic that cannot be satisfied by the LLC, still travels through the cache-slice logic portion and the ring, to the system agent • E.g., LLC misses, dirty line writeback, non-cacheable operations, and MMIO/IO operations From the Optimization Manual

  9. LLC Sharing • LLC is shared among all Cores, Graphics and Media • Graphics driver controls which streams are cached/coherent • Any agent can access all data in the LLC, independent of who allocated the line, after memory range checks • Controlled LLC way allocation mechanism prevents thrashing between Core/GFX PCI Express* DMI • Multiple coherency domains • IA Domain (Fully coherent via cross-snoops) • Graphic domain (Graphics virtual caches, flushed to IA domain by graphics engine) • Non-Coherent domain (Display data, flushed to memory by graphics engine) System Agent IMC Display Core LLC Core LLC Core LLC LLC Core Much higher Graphics performance, DRAM power savings, more DRAM BW available for Cores Graphics Foil taken from IDF 2011

  10. Cache Hierarchy • The LLC is inclusive of all cache levels above it • Data contained in the core caches must also reside in the LLC • Each LLC cache line holds an indication of the cores that may have this line in their L2 and L1 caches • Fetching data from LLC when another core has the data • Clean hit – data is not modified in the other core – 43 cycles • Dirty hit – data is modified in the other core – 60 cycles From the Optimization Manual

  11. Data Prefetch to L2$ and LLC • Two HW prefetchers fetch data from memory to L2$ and LLC • Streamer and spatial prefetcher prefetch the data to the LLC • Typically data is brought also to the L2 • Unless the L2 cache is heavily loaded with missing demand requests. • Spatial Prefetcher • Strives to complete every cache line fetched to the L2 cache with the pair line that completes it to a 128-byte aligned chunk • Streamer Prefetcher • Monitors read requests from the L1 caches for ascending and descending sequences of addresses • L1 D$ requests: loads, stores, and L1 D$ HW prefetcher • L1 I$ code fetch requests • When a forward or backward stream of requests is detected • The anticipated cache lines are pre-fetched • Prefetch-edcache lines must be in the same 4K page From the Optimization Manual

  12. Data Prefetch to L2$ and LLC • Streamer Prefetcher Enhancement • The streamer may issue two prefetch requests on every L2 lookup • Runs up to 20 lines ahead of the load request • Adjusts dynamically to the number of outstanding requests per core • Not many outstanding requests  prefetch further ahead • Many outstanding requests  prefetch to LLC only, and less far ahead • When cache lines are far ahead • Prefetch to LLC only and not to the L2$ • Avoids replacement of useful cache lines in the L2$ • Detects and maintains up to 32 streams of data accesses • For each 4K byte page, can maintain one forward and one backward stream From the Optimization Manual

  13. Lean and Mean System Agent • Contains PCI Express*, DMI, Memory Controller, Display Engine… • Contains Power Control Unit • Programmable uController, handles all power management and reset functions in the chip • Smart integration with the ring • Provides cores/Graphics /Media with high BW, low latency to DRAM/IO for best performance • Handles IO-to-cache coherency • Separate voltage and frequency from ring/cores, Display integration for better battery life • Extensive power and thermal management for PCI Express* and DDR DMI PCI Express* System Agent IMC Display Core LLC Core LLC Core LLC LLC Core Graphics Smart I/O Integration Foil taken from IDF 2011

  14. The System Agent • The system agent contains the following components • An arbiter that handles all accesses from the ring domain and from I/O (PCIe* and DMI) and routes the accesses to the right place • PCIecontrollers connect to external PCIedevices • Support different configurations: x16+x4, x8+x8+x4, x8+x4+x4+x4 • DMI controller connects to the PCH chipset • Integrated display engine, Flexible Display Interconnect, and Display Port, for the internal graphic operations • Memory controller • All main memory traffic is routed from the arbiter to the memory controller • The memory controller supports two channels of DDR • Data rates of 1066MHz, 1333MHz and 1600MHz • 8 bytes per cycle • Addresses are distributed between memory channels based on a local hash function that attempts to balance the load between the channels in order to achieve maximum bandwidth and minimum hotspot collisions From the Optimization Manual

  15. The Memory Controller • For best performance • Populate both channels with equal amounts of memory • Preferably the exact same types of DIMMs • Using more ranks for the same amount of memory, results in somewhat better memory bandwidth • Since more DRAM pages can be open simultaneously • Use highest supported speed DRAM, with the best DRAM timings • The two memory channels have separate resources • Handle memory requests independently • Each memory channel contains a 32 cache-line write-data-buffer • The memory controller contains a high-performance out-of-order scheduler • Attempts to maximize memory bandwidth while minimizing latency • Writes to the memory controller are considered completed when they are written to the write-data-buffer • The write-data-buffer is flushed out to main memory at a later time, not impacting write latency From the Optimization Manual

  16. The Memory Controller • Partial writes are not handled efficiently on the memory controller • May result in read-modify-write operations on the DDR channel • if the partial-writes do not complete a full cache-line in time • Software should avoid creating partial write transactions whenever possible and consider alternative • such as buffering the partial writes into full cache line writes • The memory controller also supports high-priority isochronous requests • E.g., USB isochronous, and Display isochronous requests • High bandwidth of memory requests from the integrated display engine takes up some of the memory bandwidth • Impacts core access latency to some degree From the Optimization Manual

  17. Integration: Optimization Opportunities • Dynamically redistribute power between Cores& Graphics • Tight power management control of all components, providing better granularity and deeper idle/sleep states • Three separate power/frequency domains: System Agent (Fixed), Cores+Ring, Graphics (Variable) • High BW Last Level Cache, shared among Cores and Graphics • Significant performance boost, saves memory bandwidth and power • Integrated Memory Controller and PCI Express ports • Tightly integrated with Core/Graphics/LLC domain • Provides low latency & low power – remove intermediate busses • Bandwidth is balanced across the whole machine, from Core/Graphics all the way to Memory Controller • Modular uArch for optimal cost/power/performance • Derivative products done with minimal effort/time Foil taken from IDF 2011

  18. DRAM

  19. Basic DRAM chip • DRAM access sequence • Put Row on addr. bus and assert RAS# (Row Addr. Strobe) to latch Row • Put Column on addr. bus and assert CAS# (Column Addr. Strobe) to latch Col • Get data on address bus Row Address decoder RAS# Row Address Latch Memory array Addr ColumnAddress Latch Column addr decoder CAS# Data

  20. DRAM Operation AL • DRAM cell consists of transistor + capacitor • Capacitor keeps the state;Transistor guards access to the state • Reading cell state: raise access line AL and sense DL • Capacitor charged  current to flow on the data line DL • Writing cell state: set DL and raise AL to charge/drain capacitor • Charging and draining a capacitor is not instantaneous • Leakage current drains capacitor even when transistor is closed • DRAM cell periodically refreshed every 64ms DL M C

  21. tRP – Row Precharge RAS# tRCD – RAS/CAS delay CAS# A[0:7] Row j X Row i Col n X CL – CAS latency Data Data n DRAM Access Sequence Timing • Put row address on address bus and assert RAS# • Wait for RAS# to CAS# delay (tRCD) between asserting RAS and CAS • Put column address on address bus and assert CAS# • Wait for CAS latency (CL) between time CAS# asserted and data ready • Row precharge time: time to close current row, and open a new row

  22. RAS# Chip select Time delay gen. A[20:23] address decoder CAS# Select A[10:19] address mux DRAM D[0:7] Memory address bus A[0:9] R/W# DRAM controller • DRAM controller gets address and command • Splits address to Row and Column • Generates DRAM control signals at the proper timing • DRAM data must be periodically refreshed • DRAM controller performs DRAM refresh, using refresh counter

  23. Improved DRAM Schemes • Paged Mode DRAM • Multiple accesses to different columns from same row • Saves RAS and RAS to CAS delay • Extended Data Output RAM (EDO RAM) • A data output latch enables to parallel next column address with current column data RAS# CAS# A[0:7] X Row X Col n X Col n+2 X Col n+1 X D n+2 Data Data n D n+1 RAS# CAS# A[0:7] X Row X Col n X Col n+2 X Col n+1 X Data n+2 Data Data n Data n+1

  24. Improved DRAM Schemes (cont) • Burst DRAM • Generates consecutive column address by itself RAS# CAS# A[0:7] Row X Col n X X Data n+2 Data Data n Data n+1

  25. Synchronous DRAM – SDRAM • All signals are referenced to an external clock (100MHz-200MHz) • Makes timing more precise with other system devices • 4 banks – multiple pages open simultaneously (one per bank) • Command driven functionality instead of signal driven • ACTIVE: selects both the bank and the row to be activated • ACTIVE to a new bank can be issued while accessing current bank • READ/WRITE: select column • Burst oriented read and write accesses • Successive column locations accessed in the given row • Burst length is programmable: 1, 2, 4, 8, and full-page • May end full-page burst by BURST TERMINATE to get arbitrary burst length • A user programmable Mode Register • CAS latency, burst length, burst type • Auto pre-charge: may close row at last read/write in burst • Auto refresh: internal counters generate refresh address

  26. clock cmd ACT NOP RD RD+PC ACT NOP ACT RD RD NOP NOP NOP NOP t RRD > 20ns t RC>70ns tRCD> 20ns Bank Bank 0 X Bank 0 Bank 0 Bank 1 X Bank 0 Bank 0 Bank 1 X X X X Addr Row i X Col j Col k Row m X Row l Col q Col n X X X X CL=2 Data n Data q Data Data j Data k SDRAM Timing • tRCD: ACTIVE to READ/WRITE gap = tRCD(MIN) / clock period • tRC: successive ACTIVE to a different row in the same bank • tRRD: successive ACTIVE commands to different banks BL = 1

  27. 0:n-1 SDRAM Array 0:n-1 0:2n-1 400Mxfer/sec n:2n-1 200MHz clock DDR-SDRAM • 2n-prefetch architecture • DRAM cells are clocked at the same speed as SDR SDRAM cells • Internal data bus is twice the width of the external data bus • Data capture occurs twice per clock cycle • Lower half of the bus sampled at clock rise • Upper half of the bus sampled at clock fall • Uses 2.5V (vs. 3.3V in SDRAM) • Reduced power consumption

  28. t RRD >20ns tRCD >20ns t RC>70ns n j +1 +1 +2 +2 +3 +3 DDR SDRAM Timing 133MHz clock cmd ACT NOP RD NOP ACT NOP NOP ACT RD NOP NOP NOP NOP Bank Bank 0 X Bank 0 X Bank 1 X X Bank 0 Bank 1 X X X X Addr Row i X Col j X Row m X X Row l Col n X X X X CL=2 Data

  29. DIMMs • DIMM: Dual In-line Memory Module • A small circuit board that holds memory chips • 64-bit wide data path (72 bit with parity) • Single sided: 9 chips, each with 8 bit data bus • Dual sided: 18 chips, each with 4 bit data bus • Data BW: 64 bits on each rising and falling edge of the clock • Other pins • Address – 14, RAS, CAS, chip select – 4, VDC – 17, Gnd – 18, clock – 4, serial address – 3, …

  30. DDR Standards • DRAM timing, measured in I/O bus cycles, specifies 3 numbers • CAS Latency – RAS to CAS Delay – RAS Precharge Time • CAS latency (latency to get data in an open page) in nsec • CAS Latency × I/O bus cycle time • Total BW for DDR400 • 3200M Byte/sec = 64 bit2200MHz / 8 (bit/byte) • 6400M Byte/sec for dual channel DDR SDRAM

  31. DDR2 • DDR2 doubles the bandwidth • 4n pre-fetch: internally read/write 4×the amount of data as the external bus • DDR2-533 cell works at the same freq. as a DDR266 cell or a PC133 cell • Prefetching increases latency • Smaller page size: 1KB vs. 2KB • Reduces activation power – ACTIVATE command reads all bits in the page • 8 banks in 1Gb densities and above • Increases random accesses • 1.8V (vs 2.5V) operation voltage • Significantly lower power Memory Cell Array I/O Buffers Data Bus Memory Cell Array Memory Cell Array I/O Buffers I/O Buffers Data Bus Data Bus

  32. DDR2 Standards

  33. DDR3 • 30% power consumption reduction compared to DDR2 • 1.5V supply voltage, compared to DDR2's 1.8V • 90 nanometer fabrication technology • Higher bandwidth • 8 bit deep prefetch buffer (vs. 4 bit in DDR2 and 2 bit in DDR) • Transfer data rate • Effective clock rate of 800–1600 MHz using both rising and falling edges of a 400–800 MHz I/O clock • DDR2: 400–800 MHz using a 200–400 MHz I/O clock • DDR: 200–400 MHz based on a 100–200 MHz I/O clock • DDR3 DIMMs • 240 pins, the same number as DDR2, and are the same size • Electrically incompatible, and have a different key notch location

  34. DDR3 Standards

  35. DDR2 vs. DDR3 Performance Source: xbitlabs The high latency of DDR3 SDRAM has negative effect on streaming operations

  36. How to get the most of Memory ? • Single Channel DDR • Dual channel DDR • Each DIMM pair must be the same • Balance FSB and memory bandwidth • 800MHz FSB provides 800MHz × 64bit / 8 = 6.4 G Byte/sec • Dual Channel DDR400 SDRAM also provides 6.4 G Byte/sec L2 Cache DRAM Ctrlr Memory Bus FSB – Front Side Bus DDR DIMM CPU CH A DDR DIMM L2 Cache DRAM Ctrlr FSB – Front Side Bus CPU CH B DDR DIMM

More Related