1 / 38

An Introduction to Cache Design

An Introduction to Cache Design. Cache. A safe place for hiding and storing things. Webster Dictionary. Even with the inclusion of cache, almost all CPUs are still mostly strictly limited by the cache access-time :

francisn
Download Presentation

An Introduction to Cache Design

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An Introduction to Cache Design \course\cpeg323-05F\Topic7a

  2. Cache A safe place for hiding and storing things. Webster Dictionary \course\cpeg323-05F\Topic7a

  3. Even with the inclusion of cache, almost all CPUs are still mostly strictly limited by the cache access-time: In most cases, if the cache access time were decreased, the machine would speedup accordingly. - Alan Smith - Even more so for MPs! \course\cpeg323-05F\Topic7a

  4. While one can imagine ref. patterns that can defeat existing cache M designs, it is the author’s experience that cache M improve performance for any program or workload which actually does useful computation. \course\cpeg323-05F\Topic7a

  5. Optimizing the design of a cache memory generally has four aspects: • Maximizing the probability of finding a memory reference’s target in the cache (the hitratio), • Minimizing the time to access information that is indeed in the cache (access time), • Minimizing the delay due to a miss, and • Minimizing the overheads of updating main memory, maintaining cache coherence etc. \course\cpeg323-05F\Topic7a

  6. . = 4 ~ 20 . = 104 ~ 106 Key Factor in Design Decision for VM and Cache Access-timeMainMem Access-timeCache Access-timeSecondaryMem Access-timeMainMem Cache control is usually implemented in hardware. \course\cpeg323-05F\Topic7a

  7. Technology in 1990s: Technology in 2000s ? \course\cpeg323-05F\Topic7a

  8. Technology in 2000s: \course\cpeg323-05F\Topic7a

  9. Secondary Memory Main Memory Processor Cache Cache in Memory Hierarchy \course\cpeg323-05F\Topic7a

  10. \course\cpeg323-05F\Topic7a

  11. Four Questions for Classifying Memory Hierarchies: The fundamental principles that drive all memory hierarchies allow us to use terms that transcend the levels we are talking about. These same principles allow us to pose four questions about any level of the hierarchy: \course\cpeg323-05F\Topic7a

  12. Four Questions for Classifying Memory Hierarchies Q1: Where can a block be placed in the upper level? (Block placement) Q2: How is a block found if it is in the upper level? (Block identification) Q3: Which block should be replaced on a miss? (Block replacement) Q4: What happens on a write? (Write strategy) \course\cpeg323-05F\Topic7a

  13. These questions will help us gain an understanding of the different tradeoffs demanded by the relationships of memories at different levels of a hierarchy. \course\cpeg323-05F\Topic7a

  14. 0 1 2 3 4 5 6 7 TAGSDATA 0117X 35, 72, 55, 30, 64, 23, 16, 14 7620X 11, 31, 26, 22, 55, … 3656X 71, 72, 44, 50, … 1741X 33, 35, 07, 65, ... Line 01173 30 ADDRESS DATA Concept of Cache miss and Cache hit \course\cpeg323-05F\Topic7a

  15. teff : effective cache access time tcache : cache access time tmain : main memory access time h : hit ratio teff = htcache + (1-h)tmain \course\cpeg323-05F\Topic7a

  16. Example Let tcache = 10 ns - 1- 4 clock cycles tmain = 50 ns - 8-32 clock cycles h = 0.95 teffect = ? 10 x 0.95 + 50 x 0.05 9.5 + 2.5 = 12 \course\cpeg323-05F\Topic7a

  17. Hit Ratio • Need high enough (say > 90%) to obtain desirable level of performance • Amplifying effect of changes • Never a constant even for the same machine \course\cpeg323-05F\Topic7a

  18. tmain tcache tmain tcache tmain tcache ~ ~ Sensitivity of Performance w.r.t h (hit ratio) teff = h tcache + (1-h) tmain = tcache [ h + (1-h) ] tcache [ 1 + (1-h) ] since 10, the magnifactor of h changes is 10 times. Conclusion: very sensitive \course\cpeg323-05F\Topic7a

  19. Remember: “h1” • Example: Let h = 0.90 if h = 0.05 (0.90 0.95) then (1 - h) = 0.05 then teff = tcache ( 1 + 0.5) ~ ~ \course\cpeg323-05F\Topic7a

  20. Basic Terminology • Cache line (block) - size of a room 1 ~ 16 words • Cache directory - key of rooms Cache may use associativity to find the “right directory” by matching “A collection of contiguous data that are treated as a single entity of cache storage.” The portion of a cache that holds the access keys that support associative access. \course\cpeg323-05F\Topic7a

  21. Cache Organization • Fully associative: an element can be in any block • Direct mapping : an element can be in only one block • Set-associative : an element can be in a group of block \course\cpeg323-05F\Topic7a

  22. An Example Mem Size = 256 k words x 4B/W = 1 MB Cache Size = 2 k words = 8 K byte Block Size = 16 word/block = 64 byte/block So Main M has = 16 K blocks (16,384) Cache has = 128 blocks addr = 18 bit + 2 = (28 x 210) x 22 256K 16 2K 16 (byte) 20 words 256 K \course\cpeg323-05F\Topic7a

  23. Fully Associative Feature • any block in M can be in any block-frame in cache • all entries (block frame) are compared simultaneously (by associative search) \course\cpeg323-05F\Topic7a

  24. A Special Case simplest example: a block = a word entire memory word address becomes “tag” 0 17 Address 027560 very “flixible” and higher probability to reside in cache. 0 17 Cache adv: no trashing (quick reorganizing) disadv: overhead of associative search: cost + time 027560 data \course\cpeg323-05F\Topic7a

  25. Block 0 Block 1 Block I Block 16382 Block 16383 14 bits Tag Block 0 Block 1 Block 127 . . . Tag … … … … … ... Tag . . . Tag 14 4 Main memory address tag word Recall: each block has 16 word – so you need 4 bits Fully associative cache organization \course\cpeg323-05F\Topic7a

  26. Direct Mapping • No associative match • From M-addr, “directly” indexed to the block frame in cache where the block should be located. A comparison then is to used to determine if it is a miss or hit. \course\cpeg323-05F\Topic7a

  27. Direct Mapping Cont’d Advantage: simplest: Disadvantage: “trashing” Fast (fewer logic) Low cost: (only one set comparator is needed hence can be in the form of standard M \course\cpeg323-05F\Topic7a

  28. Direct Mapping Cont’d • Example: since cache only has 128 block frames so the degree of multiplexing: • Disadr: “trashing” Main Memory Size 16384 (block) 128 (27) 128 for addressing the corresponding frame or set of size 1. = = 27 block/frame the high-order 7 bit is used as tag. i.e. 27 blocks “fall” in one block frame. \course\cpeg323-05F\Topic7a

  29. Main memory Cache Block 0 Block 1 Block 2 Block 127 Block 128 Block 129 Block 255 Block 256 Block 257 Block 4095 Block 4096 Block 16383 7 bits Tag ... Block 0 Block 1 Block 127 Tag ... … … … … … ... Tag ... Tag ... Main memory address 7 7 4 Tag Block Word Direct Mapping \course\cpeg323-05F\Topic7a

  30. Direct Mapping Cont’d Mapping (indexing) block addr mod (# of blocks in cache – in this case: mod (27)) Adv: low-order log2 (cache size) bit can be used for indexing \course\cpeg323-05F\Topic7a

  31. M S Set-Associative • A compromises between direct/full-associative • The cache is divided into S sets S = 2, 4, 8, … • If the cache has M blocks than, all together, there are E = blocks/set # of buildings available for indexing In our example, S = 128/2 = 64 sets \course\cpeg323-05F\Topic7a

  32. Tag Tag Set 1 Tag Set 63 Tag Main memory address 8 6 4 Tag Set Word The 6-bit will index to the right set, then the 8-bit tag will be used for an associative match. Main memory Cache Block 0 Block 1 Block 63 Block 64 Block 65 Block 4095 Block 16383 8 bits ... Tag Block 0 Block 1 Block 2 Block 3 Block 126 Block 127 Set 0 ... Tag \course\cpeg323-05F\Topic7a

  33. 214 (16k) 26 = 28 block/set 28 block/per set of 2 blocks a 2-way set associative organization: 8 6 4 2 Set Word thus or available for indexing 6 bit used to index into the right “set” higher order 2 way 8 bit used as tag hence an associative match of 8 bit with the tags of the 2 blocks is required Hence an associative matching of 8 bit with the tags of the 2 block is required. \course\cpeg323-05F\Topic7a

  34. 0 6 7 13 14 17 7 7 4 Sector block word (tag) Sector Mapping Cache • Sector (IBM 360/85) - 16 sector x 16 block/sector • 1 sector = consecutive multiple blocks • Cache miss: sector replacement • Valid bit - one block is moved on demand • Example: A sector in memory can be in any sector in cache \course\cpeg323-05F\Topic7a

  35. Tag Main memory address 10 4 4 Sector Block Word (tag) Valid bit Block 0 Block 1 Block 15 Block 16 Block 31 Block 16368 Block 16383 Sector 0 Sector 0 ... Tag Block 0 Block 1 Block 14 Block 15 Block 16 Block 31 Block 112 Block 127 Sector 0 ... Sector 1 ... Sector 1 . . . Tag Sector 7 ... ... Sector 1023 Sector mapping cache \course\cpeg323-05F\Topic7a

  36. 16k 16 cont’d 128 blocks 16 blocks/sector Cache has = 8 sector Main memory has = 1K sectors Sector mapping cache \course\cpeg323-05F\Topic7a

  37. 20 32 20 = Address (showing bit positions) 31 30 29 28 27……..16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 Byte offset 10 Tag Hit Data Index Valid Tag Data 0 1 2 . . . . . . . . . 1021 1022 1023 MIPS Example \course\cpeg323-05F\Topic7a

  38. Total # of Bits in a Cache Total # of bits = (# of bits of a tag + # of bits of a block + # of bits in valid field) x Cache size For a MIPS example : = ((32-14-2) + 32 + 1) x 214 = 214 x 49 = 784 k bits ~ 100 kbytes = 64 K (bytes) = 214 blocks with Assuming a directly-mapped cache. \course\cpeg323-05F\Topic7a

More Related