1 / 15

Computer Organization CS224

Computer Organization CS224. Fall 2012 Lessons 45 & 46. The Memory Hierarchy. Common principles apply at all levels of the memory hierarchy (caches, TLB, virtual memory) Based on notions of caching At each level in the hierarchy Block placement Finding a block Replacement on a miss

carol-ferguson
Download Presentation

Computer Organization CS224

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computer OrganizationCS224 Fall 2012 Lessons 45 & 46

  2. The Memory Hierarchy • Common principles apply at all levels of the memory hierarchy (caches, TLB, virtual memory) • Based on notions of caching • At each level in the hierarchy • Block placement • Finding a block • Replacement on a miss • Write policy The BIG Picture §5.5 A Common Framework for Memory Hierarchies

  3. Block Placement • Determined by associativity • Direct mapped =1-way associative (1 block per set) • One choice for placement • n-way set associative (n blocks per set) • n choices within a set • Fully associative (1 set, all cache blocks in that set) • Any location • Higher associativity reduces miss rate (e.g. Fig 5.30) • Increases complexity, cost, and access time

  4. Finding a Block • Hardware caches • Reduce comparisons to reduce cost • Virtual memory • Full table lookup makes full associativity feasible • Benefit in reduced miss rate

  5. Replacement (on a miss) • Caches (with associativity > 1) • Least recently used (LRU) • Complex and costly hardware for high associativity • Random • Close to LRU, easier to implement • Virtual memory (fully associative) • LRU approximation with hardware support

  6. Write Policy • Write-through • Update both upper and lower levels • Simplifies replacement, but may require write buffer • Write-back • Update upper level only • Update lower level when block is replaced • Need to keep more state (e.g. dirty bit) • Virtual memory • Only write-back is feasible, given disk write latency • Lowest level of cache • Generally uses write-back, since rate of CPU stores > DRAM write rate [sw: 7.6%, sh: 0.1%, sb: 0.6% for SPECint2006] • Main memory generally can’t write an item every 1/8.3% = 12 instructions, so there is no write buffer size which will allow write-through to work

  7. Sources of Misses • Compulsory misses (aka cold start misses) • First access to a block • Capacity misses • Due to finite cache size • A replaced block is later accessed again • Conflict misses (aka collision misses) • In a non-fully associative cache • Due to competition for entries in a set • Would not occur in a fully associative cache of the same total size

  8. Cache Design Trade-offs See Figure 5.30, p. 519 and Figure 5.31, p. 524

  9. Multilevel On-Chip Caches Intel Nehalem 4-core processor §5.10 Real Stuff: The AMD Opteron X4 and Intel Nehalem Per core: 32KB L1 I-cache, 32KB L1 D-cache, 512KB L2 cache

  10. 2-Level TLB Organization

  11. 3-Level Cache Organization

  12. Miss Penalty Reduction • Return requested word first • Then back-fill rest of block • Non-blocking cache • Used with out-of-order processors • Hides miss latency, allowing processing to continue • Hit under miss: allow hits to proceed • Miss under miss: allow multiple outstanding misses • Hardware pre-fetch: instructions and data • Predicts in HW, works best for data arrays in loops • Opteron X4: bank-interleaved L1 D-cache • 8 banks, 2 concurrent 128-bit accesses per cycle • Bank-interleaved memory is much cheaper than multiported DRAM See Figure 5.40, p. 532

  13. Pitfalls • Byte vs. word addressing. • Example: 32-byte direct-mapped cache, 4-byte blocks, 8 blocks • Byte 36 maps to block 1 (addr 100100, index 001) • Word 36 maps to block 4 (addr 100100, index 100) • Ignoring memory system effects when writing or generating code • Example: iterating over rows vs. columns of arrays (p. 544 code) • Large strides (decrements/increments) result in poor locality • Optimizing compilers (and good code-writers) re-organize programs, to increase spatial and temporal locality §5.11 Fallacies and Pitfalls

  14. Pitfalls • In multiprocessor with shared L2 or L3 cache • Less associativity than cores results in conflict misses • More cores  need to increase associativity • Using AMAT to evaluate performance of out-of-order processors • Ignores effect of non-blocked accesses • Instead, evaluate performance by simulation

  15. Concluding Remarks • Fast memories are small, large memories are slow • We really want fast, large memories  • Caching gives this illusion  • Principle of locality • Programs use a small part of their memory space frequently • Memory hierarchy • L1 cache  L2 cache  …  DRAM memory disk • Memory system design is critical for multiprocessors §5.12 Concluding Remarks

More Related