1 / 100

Based on slides from D. Patterson and www-inst.eecs.berkeley/~cs152/

COM 249 – Computer Organization and Assembly Language Chapter 5 Memory. Based on slides from D. Patterson and www-inst.eecs.berkeley.edu/~cs152/. Chapter 5. Large and Fast: Exploiting Memory Hierarchy. Memory Hierarchy. §5.1 Introduction.

dasan
Download Presentation

Based on slides from D. Patterson and www-inst.eecs.berkeley/~cs152/

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. COM 249 – Computer Organization andAssembly LanguageChapter 5 Memory Based on slides from D. Patterson and www-inst.eecs.berkeley.edu/~cs152/ Modified by S. J. Fritz Spring 2009 (1)

  2. Chapter 5 Large and Fast: Exploiting Memory Hierarchy Modified by S. J. Fritz Spring 2009 (2)

  3. Memory Hierarchy §5.1 Introduction • A Memory Hierarchy consists of multiple levels of memory with different speeds and sizes. Faster memories are more expensive per bit and are smaller. • 3 Primary memory technologies: • DRAM (dynamic random access memory) – Main memory • SRAM ( static random access memory) (cache) • Magnetic disk ( or more recently flash memory) Modified by S. J. Fritz Spring 2009 (3)

  4. Memory Hierarchy • Memory is designed to take advantage of the 3 basic principles: • Make the common case fast • Principle of Locality • Smaller is faster • Recently accessed items are kept in memory closest to the CPU Modified by S. J. Fritz Spring 2009 (4)

  5. Memory Hierarchy Design Memory CPU bus I/O bus I/O Devices Cache Main Memory Register file http://www.cs.iastate.edu/~prabhu/Tutorial/CACHE/mem_title.html Modified by S. J. Fritz Spring 2009 (5)

  6. Memory Technology • Static RAM (SRAM) • (0.5ns – 2.5ns), $2000 – $5000 per GB • Dynamic RAM (DRAM) • (50ns – 70ns), $20 – $75 per GB • Magnetic disk • (5ms – 20ms), $0.20 – $2 per GB • Ideal memory • Access time of SRAM • Capacity and cost/GB of disk Modified by S. J. Fritz Spring 2009 (6)

  7. Illusion of Large Memory • Programmers have always wanted large memories. • We create the illusion of large memory that we can access as fast as small memory • Library research example (p.452) • Principle of Locality- programs access a relatively small portion of their address space at any one time • Temporal locality • Spatial locality Modified by S. J. Fritz Spring 2009 (7)

  8. Principle of Locality • Programs access a small proportion of their address space at any time • Temporal locality • Items accessed recently are likely to be accessed again soon • e.g., instructions in a loop, induction variables • Spatial locality • Items near those accessed recently are likely to be accessed soon • E.g., sequential instruction access, array data Modified by S. J. Fritz Spring 2009 (8)

  9. Taking Advantage of Locality • Memory hierarchy • Store everything on disk • Copy recently accessed (and nearby) items from disk to smaller DRAM memory • Main memory • Copy more recently accessed (and nearby) items from DRAM to smaller SRAM memory • Cache memory attached to CPU Modified by S. J. Fritz Spring 2009 (9)

  10. Memory Hierarchy Levels • Level closer to processor is subset of a level farther away • Block (or line): unit of copying • May be multiple words • If accessed data is present in upper level • Hit: access satisfied by upper level • Hit ratio: hits/accesses • If accessed data is absent • Miss: block copied from lower level • Time taken: miss penalty • Miss ratio: misses/accesses= 1 – hit ratio • Then accessed data supplied from upper level Modified by S. J. Fritz Spring 2009 (10)

  11. Memory Hierarchy CPU volatile non-volatile Modified by S. J. Fritz Spring 2009 (11)

  12. Hierarchy: Cost, Speed, Size http://en.wikipedia.org/wiki/Memory_hierarchy Modified by S. J. Fritz Spring 2009 (12)

  13. Cache Memory §5.2 The Basics of Caches • A cache is a safe place to store things (like the desk in the library example) • Cache was the level of memory between processor and main memory in early computers. • Today, cache refers to any memory storage that takes advantage of locality of access. • Simple cache – processor requests are one word and blocks are one word. Modified by S. J. Fritz Spring 2009 (13)

  14. Direct Mapped Cache • Location determined by address • Direct mapped: only one choice- each location in memory is mapped to only one location in cache • (Block address) modulo (#Blocks in cache) • # Blocks is a power of 2 • Use low-order address bits • Since there are 8 cache blocks, map X to cache word mod 8 Modified by S. J. Fritz Spring 2009 (14)

  15. Tags and Valid Bits • How do we know which particular block is stored in a cache location? • Store block address as well as the data • Actually, only need the high-order bits • Called the tag • What if there is no data in a location? • Valid bit: 1 = present, 0 = not present • Initially 0 • Reference address • Tag field – compared with cache • Cache index- used to select the block Modified by S. J. Fritz Spring 2009 (15)

  16. Cache Access • 8-blocks, 1 word/block, direct mapped • Initial state - initially empty will cause a cache “miss” when accessed. Modified by S. J. Fritz Spring 2009 (16)

  17. Cache Example Processor request Modified by S. J. Fritz Spring 2009 (17)

  18. Cache Example Processor request Modified by S. J. Fritz Spring 2009 (18)

  19. Cache Example Modified by S. J. Fritz Spring 2009 (19)

  20. Cache Example Modified by S. J. Fritz Spring 2009 (20)

  21. Cache Replace What happens when we try to access cache block 010? Modified by S. J. Fritz Spring 2009 (21)

  22. Cache Replacement 18 replaces previous value 26 Modified by S. J. Fritz Spring 2009 (22)

  23. Address Subdivision Modified by S. J. Fritz Spring 2009 (23)

  24. 31 10 9 4 3 0 Tag Index Offset 22 bits 6 bits 4 bits Example: Larger Block Size • 64 blocks, 16 bytes/block • To what block number does address 1200 map? • Block address = 1200/16 = 75 • Block number = 75 modulo 64 = 11 • Address 1200 maps to cache block 11. Address format Modified by S. J. Fritz Spring 2009 (24)

  25. Block Size Considerations • Larger blocks should reduce miss rate • Due to spatial locality • But in a fixed-sized cache • Larger blocks  fewer of them • More competition  increased miss rate • Larger blocks  pollution • Larger miss penalty • Can override benefit of reduced miss rate • Early restart and critical-word-first can help • Miss penalty is time to access from a lower level and load it into cache (both latency and transfer time) Modified by S. J. Fritz Spring 2009 (25)

  26. Cache Misses • On cache hit, CPU proceeds normally • On cache miss • Stall the CPU pipeline • Fetch block from next level of hierarchy • Instruction cache miss • Restart instruction fetch • Data cache miss • Complete data access Modified by S. J. Fritz Spring 2009 (26)

  27. Handling Writes • Write through • Writes always update both the cache and the next lower level memory • Ensures that data is always consistent • Write back • Updates values only to the block in the cache, then writes the modified block to the cache when the block is replaced • What are the advantages of each? Modified by S. J. Fritz Spring 2009 (27)

  28. Write-Through • On data-write hit, could just update the block in cache • But then cache and memory would be inconsistent • Write through: also update memory • But makes writes take longer • e.g., if base CPI = 1, 10% of instructions are stores, write to memory takes 100 cycles • Effective CPI = 1 + 0.1×100 = 11 • Solution: write buffer • Holds data waiting to be written to memory • CPU continues immediately • Only stalls on write if write buffer is already full Modified by S. J. Fritz Spring 2009 (28)

  29. Write-Back • Alternative: On data-write hit, just update the block in cache • Keep track of whether each block is dirty • When a dirty block is replaced • Write it back to memory • Can use a write buffer to allow replacing block to be read first Modified by S. J. Fritz Spring 2009 (29)

  30. Write Allocation • What should happen on a write miss? • Alternatives for write-through • Allocate on miss: fetch the block • Write around: don’t fetch the block • Since programs often write a whole block before reading it (e.g., initialization) • For write-back • Usually fetch the block Modified by S. J. Fritz Spring 2009 (30)

  31. Example: Intrinsity FastMATH • Embedded MIPS processor • 12-stage pipeline • Instruction and data access on each cycle • Split cache: separate I-cache and D-cache • Each 16KB: 256 blocks × 16 words/block • D-cache: write-through or write-back • SPEC2000 miss rates • I-cache: 0.4% • D-cache: 11.4% • Weighted average: 3.2% Modified by S. J. Fritz Spring 2009 (31)

  32. Example: Intrinsity FastMATH Modified by S. J. Fritz Spring 2009 (32)

  33. Main Memory Supporting Caches • Use DRAMs for main memory • Fixed width (e.g., 1 word) • Connected by fixed-width clocked bus • Bus clock is typically slower than CPU clock • Example cache block read • 1 bus cycle for address transfer • 15 bus cycles per DRAM access • 1 bus cycle per data transfer • For 4-word block, 1-word-wide DRAM • Miss penalty = 1 + 4×15 + 4×1 = 65 bus cycles • Bandwidth = 16 bytes / 65 cycles = 0.25 B/cycle Modified by S. J. Fritz Spring 2009 (33)

  34. Designing the Memory System Three options: • Memory is one word wide and all accesses are sequential • Widen memory and bus for parallel access to multiple words • Increase the bandwidth by widening the connection, but not the bus – allows all to be read simultaneously. Called interleaving (See diagram on next slide) Modified by S. J. Fritz Spring 2009 (34)

  35. Increasing Memory Bandwidth • 4-word wide memory • Miss penalty = 1 + 15 + 1 = 17 bus cycles • Bandwidth = 16 bytes / 17 cycles = 0.94 B/cycle • 4-bank interleaved memory • Miss penalty = 1 + 15 + 4×1 = 20 bus cycles • Bandwidth = 16 bytes / 20 cycles = 0.8 B/cycle Modified by S. J. Fritz Spring 2009 (35)

  36. Advanced DRAM Organization • Bits in a DRAM are organized as a rectangular array • DRAM accesses an entire row • Burst mode: supply successive words from a row with reduced latency • Double data rate (DDR) DRAM • Transfer on rising and falling clock edges • Quad data rate (QDR) DRAM • Separate DDR inputs and outputs Modified by S. J. Fritz Spring 2009 (36)

  37. DRAM Generations Modified by S. J. Fritz Spring 2009 (37)

  38. Measuring Cache Performance Techniques for improving cache performance • Reduce the miss rate • reduce the probability that two different memory blocks will contend for the same cache location • Reduce the miss penalty • Add additional level to the hierarchy • Called multilevel caching • First in 1990 in machines > $100,000 • Now common on desktops §5.3 Measuring and Improving Cache Performance Modified by S. J. Fritz Spring 2009 (38)

  39. Measuring Cache Performance • Components of CPU time • Program execution cycles • Includes cache hit time • Memory stall cycles • Mainly from cache misses • With simplifying assumptions: Modified by S. J. Fritz Spring 2009 (39)

  40. Cache Performance Example • Given • I-cache miss rate = 2% • D-cache miss rate = 4% • Miss penalty = 100 cycles • Base CPI (ideal cache) = 2 • Load & stores are 36% of instructions • Miss cycles per instruction • I-cache: 0.02 × 100 = 2 • D-cache: 0.36 × 0.04 × 100 = 1.44 • Actual CPI = 2 + 2 + 1.44 = 5.44 • Ideal CPU is 5.44/2 =2.72 times faster Modified by S. J. Fritz Spring 2009 (40)

  41. Average Access Time • Hit time is also important for performance • Average memory access time (AMAT) • AMAT = Hit time + Miss rate × Miss penalty • Example • CPU with 1ns clock, hit time = 1 cycle, miss penalty = 20 cycles, I-cache miss rate = 5% • AMAT = 1 + 0.05 × 20 = 2ns • 2 cycles per instruction Modified by S. J. Fritz Spring 2009 (41)

  42. Performance Summary • When CPU performance increased • Miss penalty becomes more significant • Decreasing base CPI • Greater proportion of time spent on memory stalls • Increasing clock rate • Memory stalls account for more CPU cycles • Can’t neglect cache behavior when evaluating system performance Modified by S. J. Fritz Spring 2009 (42)

  43. Flexible Block Placement A block can go in exactly one place in the cache: • Direct mapped – there is a direct mapping form any block address to a single location • Fully associative – a block in memory may be associated with any entry in the cache – to find it search all entries in parallel • Set Associative – there are a fixed number (n) of locations where a block can be placed. Each block maps to a unique set given by the index field, and a block can be in any element of that set. Modified by S. J. Fritz Spring 2009 (43)

  44. Associative Caches • Fully associative • Allow a given block to go in any cache entry • Requires all entries to be searched at once • All tags must be searched • Comparator per entry (expensive) • n-way set associative • Each set contains n entries • Block number determines which set • (Block number) modulo (#Sets in cache) • Search all entries in a given set at once • All tage in the set are searched • n comparators (less expensive) Modified by S. J. Fritz Spring 2009 (44)

  45. Associative Cache Example Modified by S. J. Fritz Spring 2009 (45)

  46. Spectrum of Associativity • For a cache with 8 entries Modified by S. J. Fritz Spring 2009 (46)

  47. Associativity Example • Compare 4-block caches • Direct mapped, 2-way set associative,fully associative • Block access sequence: 0, 8, 0, 6, 8 • Direct mapped Modified by S. J. Fritz Spring 2009 (47)

  48. Associativity Example • 2-way set associative • Fully associative Modified by S. J. Fritz Spring 2009 (48)

  49. How Much Associativity • Increased associativity decreases miss rate • But with diminishing returns • Simulation of a system with 64KBD-cache, 16-word blocks, SPEC2000 • 1-way: 10.3% • 2-way: 8.6% • 4-way: 8.3% • 8-way: 8.1% Modified by S. J. Fritz Spring 2009 (49)

  50. Set Associative Cache Organization Modified by S. J. Fritz Spring 2009 (50)

More Related