Memory System Performance

1. Memory System Performance What affects memory performance? Hit Time : Time to send data from the cache to the CPU Miss Penalty : Time to copy data from memory into the cache Miss Rate : Percentage of memory accesses to locations outside cache

2. Average Memory Access Time (AMAT) AMAT = (Hit rate x Hit time) + (Miss rate x Miss time) Miss time = Miss penalty + Hit time So, AMAT = Hit time + (Miss rate x Miss penalty) • Hit rate + Miss rate = 100% • We always read from the cache, even in the case of a cache miss.

3. Memory Address 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111 Index 00 01 10 11 Direct Mapped Cache • Good: easy • Bad: Simple access patterns (2, 6, 2, 6, 2, …) mean lots of cache misses!

4. Memory Address Memory Address 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111 Index Index 000 001 010 011 100 101 110 111 00 01 10 11 Eliminate simple-pattern misses…

5. Memory Address 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111 Index 00 01 10 11 …won’t work! Memory Address 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111 Index 000 001 010 011 100 101 110 111

6. Memory Address 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111 00 01 10 11 Index Index 00 01 10 11 Another way: two caches…

7. 00 01 10 11 …but its really one cache Memory Address 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111 Index Each block: two possible locations

8. Memory Address 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111 Index 00 01 10 11 This is what we mean by a Set Two-way set associative

9. Memory Address 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111 Index 00 01 10 11 “Flat” sets (book)

10. 00 01 10 11 “Bucket” sets (lecture slides) Memory Address Index 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111

11. Connections • If the cache can hold B blocks of data in total… • direct mapped: blocks per sets, sets • 2-way: blocks per set, sets • 4-way: blocks per set, sets • Fully associative: blocks per set, sets • Index: • Tag: • Block offset: 1 B 2 B / 2 4 B / 4 B 1 Which set contains the data (if in cache) To verify if block really is in the set Which byte within the block contains the data

12. Address (m bits) Block offset Tag Index (m-k-n) k Time (lots of hardware) Index Valid Tag Data Valid Tag Data 0 ... 2k 2n 2n = = 2-to-1 mux 2n Hit Data Full associativity is expensive! • Reason 1: • Reason 2: No index field, tag = entire address (lots of overhead)

13. Least Recently Used (LRU) • What if all blocks are used? • Eliminate least recently used • if I haven’t used it recently, I won’t need it soon

14. (m-s-n) s n Block offset Address (m bits) Tag Index Locating a set associative block • 2s sets, 2n bytes per block • Index selects the set, not an individual block Block Offset = Memory Address mod 2n Block Address = Memory Address / 2n Set Index = Block Address mod 2s