1 / 76

Caching

Explore cache design principles, memory hierarchy, and program characteristics to create efficient systems. Learn about spatial and temporal locality, data placement strategies, and direct-mapped caches for optimizing memory usage.

johnjjohnson
Download Presentation

Caching

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Caching Chapter 7

  2. Memory Hierarchy Size Smallest Largest Cost/bit Highest Lowest Tech SRAM (logic) SRAM (logic) DRAM (capacitors) Speed Fastest Slowest CPU L1 L2 Cache DRAM

  3. Two design decisions • What shall we put in the cache? • How shall we organize cache to • find things quickly • hold the most important data • freezer or backpack….

  4. What to put in cache?Try to apply a similar problem’s solution • Can we predict what data we will use?

  5. What to put in cache? • Can we predict what data we will use? • Instead of predicting branch direction, predict next memory address request

  6. What to put in cache? • Can we predict what data we will use? • Instead of predicting branch direction, predict next memory address request • Like branch prediction, use previous behavior

  7. What to put in cache? • Can we predict what data we will use? • Instead of predicting branch direction, predict next memory address request • Like branch prediction, use previous behavior • Keep a prediction for every load? • Fetch stage for load is *TOO LATE* • Keep a prediction per-memory address?

  8. What to put in cache? • Can we predict what data we will use? • Instead of predicting branch direction, predict next memory address request • Like branch prediction, use previous behavior • Keep a prediction for every load? • Fetch stage for load is *TOO LATE* • Keep a prediction per-memory address? • Given address, guess next likely address

  9. What to put in cache? • Can we predict what data we will use? • Instead of predicting branch direction, predict next memory address request • Like branch prediction, use previous behavior • Keep a prediction for every load? • Fetch stage for load is *TOO LATE* • Keep a prediction per-memory address? • Given address, guess next likely address • Too many choices – table too large or fits too few

  10. Program CharacteristicsFind out more about programs • Temporal Locality • Spatial Locality

  11. Program Characteristics • Temporal Locality • If you use one item, you are likely to use it again soon • Spatial Locality

  12. Program Characteristics • Temporal Locality • If you use one item, you are likely to use it again soon • Spatial Locality • If you use one item, you are likely to use its neighbors soon

  13. Locality • Programs tend to exhibit spatial & temporal locality. Just a fact of life. • How can we use this knowledge of program behavior to design a cache?

  14. What does that mean?!? • 1. Design cache that takes advantage of spatial & temporal locality

  15. What does that mean?!? • 1. Design cache that takes advantage of spatial & temporal locality • 2. When you program, place data together that is used together to increase spatial & temporal locality

  16. What does that mean?!? • 1. Design cache that takes advantage of spatial & temporal locality • 2. When you program, place data together that is used together to increase locality • Java - difficult to do • C - more control over data placement • Note: Caches exploit locality. Programs have varying degrees of locality. Caches do not have locality!

  17. Cache Design • Temporal Locality • Spatial Locality

  18. Cache Design • Temporal Locality • When we obtain the data, store it in the cache. • Spatial Locality

  19. Cache Design • Temporal Locality • When we obtain the data, store it in the cache. • Spatial Locality • Transfer large block of contiguous data to get item’s neighbors. • Block (Line): Amount of data transferred for a single miss (data plus neighbors)

  20. Where do we put data? • Searching whole cache takes time & power • Direct-mapped • Limit each piece of data to one possible position • Search is quick and simple

  21. What is our “key” for lookup? • Tools are sorted by tool-type • Books are sorted by subject (Dewey-Decimal) • Old LISP machine sorted by data type • Modern machines have no information – can only sort by address

  22. Direct-Mapped Each box corresponds to one word (4 bytes) 000000 000100 Index 010000 00 010100 01 10 11 100000 100100 Cache 110000 110100 Memory

  23. Direct-Mapped 000000 000100 One block (line) Index 010000 00 010100 01 10 11 100000 100100 Cache 110000 110100 Memory

  24. Draw on the board!!! Show what addresses go where Direct-Mapped 000000 000100 One block (line) Index 010000 00 010100 01 10 11 100000 100100 Cache 110000 110100 Memory

  25. Direct-Mapped cacheBlock (Line) size = 2 words or 8 bytes Byte Address 0b100100100 Index Data 00 01 10 11 Where do we look in the cache? How do we know if it is there?

  26. Direct-Mapped cacheBlock (Line) size = 2 words or 8 bytes Byte Address 0b100100100 Index Data 00 01 Where is it within the block? 10 Block Address 11 Where do we look in the cache? BlockAddress mod #sets BlockAddress & (#sets-1) How do we know if it is there?

  27. M[288-291] Direct-Mapped cacheBlock (Line) size = 2 words or 8 bytes Byte Address 0b100100100 Valid Tag Data 00 1 1001 M[292-295] 01 Where is it within the block? 10 Index Tag 11 Where do we look in the cache? BlockAddress mod #slots BlockAddress & (#slots-1) How do we know if it is there? We need a tag & valid bit

  28. Splitting the Address Direct-Mapped Cache Valid Tag Data 0b1010001 00 0 01 0 10 0 Tag 11 0 Index Block Offset Byte Offset

  29. Definitions • Byte Offset: Which _____ within _____? • Block Offset: Which _____ within ______? • Set: Group of ______ checked each access • Index: Which ______ within cache? • Tag: Is this the right one?

  30. Definitions • Byte Offset: Which byte within word • Block Offset: Which _____ within ______? • Set: Group of ______ checked each access • Index: Which ______ within cache? • Tag: Is this the right one?

  31. Definitions • Byte Offset: Which byte within word • Block Offset: Which word within block • Set: Group of ______ checked each access • Index: Which ______ within cache? • Tag: Is this the right one?

  32. Definitions • Byte Offset: Which byte within word • Block Offset: Which word within block • Set: Group of blocks checked each access • Index: Which ______ within cache? • Tag: Is this the right one?

  33. Definitions • Byte Offset: Which byte within word • Block Offset: Which word within block • Set: Group of blocks checked each access • Index: Which set within cache? • Tag: Is this the right one? (All of the upper bits)

  34. Definitions • Block (Line) • Hit • Miss • Hit time / Access time • Miss Penalty

  35. Definitions • Block - unit of data transfer – bytes/words • Hit • Miss • Hit time / Access time • Miss Penalty

  36. Definitions • Block - unit of data transfer – bytes/words • Hit - data found in this cache • Miss • Hit time / Access time • Miss Penalty

  37. Definitions • Block - unit of data transfer – bytes/words • Hit - data found in this cache • Miss - data not found in this cache • Send request to lower level • Hit time / Access time • Miss Penalty

  38. Definitions • Block - unit of data transfer – bytes/words • Hit - data found in this cache • Miss - data not found in this cache • Send request to lower level • Hit time / Access time • Time to access this cache – look for item, return data • Miss Penalty

  39. Definitions • Block - unit of data transfer – bytes/words • Hit - data found in this cache • Miss - data not found in this cache • Send request to lower level • Hit time / Access time • Time to access this cache • Miss Penalty • Time to receive block from lower level • Not always constant

  40. Example 1 – Direct-MappedBlock size=2 words Direct-Mapped Cache Valid Tag Data 00 0 0x1010001 01 0 10 0 Tag 11 0 Index Block Offset Byte Offset

  41. Example 1 – Direct-MappedBlock size=2 words Reference Stream: Hit/Miss 0b1001000 0b0010100 0b0111000 0b0010000 0b0010100 0b0100100 Direct-Mapped Cache Valid Tag Data 00 0 01 0 10 0 11 0 Tag Byte Offset Index Block Offset Miss Rate:

  42. Example 1 – Direct-MappedBlock size=2 words Reference Stream: Hit/Miss 0b1001000 0b0010100 0b0111000 0b0010000 0b0010100 0b0100100 Direct-Mapped Cache Valid Tag Data 00 0 01 0 10 0 11 0 Tag Byte Offset Index Block Offset Miss Rate:

  43. Example 1 – Direct-MappedBlock size=2 words Reference Stream: Hit/Miss 0b1001000 M 0b0010100 0b0111000 0b0010000 0b0010100 0b0100100 Direct-Mapped Cache Valid Tag Data 00 0 01 1 10 M[76-79] M[72-75] 10 0 11 0 Tag Byte Offset Index Block Offset Miss Rate:

  44. Example 1 – Direct-MappedBlock size=2 words Reference Stream: Hit/Miss 0b1001000 M 0b0010100 0b0111000 0b0010000 0b0010100 0b0100100 Direct-Mapped Cache Valid Tag Data 00 0 01 1 10 M[76-79] M[72-75] 10 0 11 0 Tag Byte Offset Index Block Offset Miss Rate:

  45. Example 1 – Direct-MappedBlock size=2 words Reference Stream: Hit/Miss 0b1001000 M 0b0010100 0b0111000 0b0010000 0b0010100 0b0100100 Direct-Mapped Cache Valid Tag Data 00 0 01 1 10 M[76-79] M[72-75] 10 1 00 M[20-23] M[16-19] 11 0 Tag Byte Offset Index Block Offset Miss Rate:

  46. Example 1 – Direct-MappedBlock size=2 words Reference Stream: Hit/Miss 0b1001000 M 0b0010100 M 0b0111000 0b0010000 0b0010100 0b0100100 Direct-Mapped Cache Valid Tag Data 00 0 01 1 10 M[76-79] M[72-75] 10 1 00 M[20-23] M[16-19] 11 0 Tag Byte Offset Index Block Offset Miss Rate:

  47. Example 1 – Direct-MappedBlock size=2 words Reference Stream: Hit/Miss 0b1001000 M 0b0010100 M 0b0111000 M 0b0010000 0b0010100 0b0100100 Direct-Mapped Cache Valid Tag Data 00 0 01 1 10 M[76-79] M[72-75] 10 1 00 M[20-23] M[16-19] 11 1 01 M[60-63] M[56-59] Tag Byte Offset Index Block Offset Miss Rate:

  48. Example 1 – Direct-MappedBlock size=2 words Reference Stream: Hit/Miss 0b1001000 M 0b0010100 M 0b0111000 M 0b0010000 0b0010100 0b0100100 Direct-Mapped Cache Valid Tag Data 00 0 01 1 10 M[76-79] M[72-75] 10 1 00 M[20-23] M[16-19] 11 1 01 M[60-63] M[56-59] Tag Byte Offset Index Block Offset Miss Rate:

  49. Example 1 – Direct-MappedBlock size=2 words Reference Stream: Hit/Miss 0b1001000 M 0b0010100 M 0b0111000 M 0b0010000 H 0b0010100 0b0100100 Direct-Mapped Cache Valid Tag Data 00 0 01 1 10 M[76-79] M[72-75] 10 1 00 M[20-23] M[16-19] 11 1 01 M[60-63] M[56-59] Tag Byte Offset Index Block Offset Miss Rate:

  50. Example 1 – Direct-MappedBlock size=2 words Reference Stream: Hit/Miss 0b1001000 M 0b0010100 M 0b0111000 M 0b0010000 H 0b0010100 0b0100100 Direct-Mapped Cache Valid Tag Data 00 0 01 1 10 M[76-79] M[72-75] 10 1 00 M[20-23] M[16-19] 11 1 01 M[60-63] M[56-59] Tag Byte Offset Index Block Offset Miss Rate:

More Related