CSCI206 - Computer Organization & Programming

1. CSCI206 - Computer Organization & Programming Associative Cache Revised by Alexander Fuchsberger and Xiannong Meng in spring 2019 based on the notes by other instructors. zyBook: 12.4

2. 3 Causes of cache misses

3. 3 Causes of cache misses • Compulsory • The first time a block is accessed it will never be in the cache. • Capacity • The program is accessing more data than will fit in the cache (e.g., a 4 KB cache processing a 16 KB array). • Conflict • The program is using data stored at different memory locations but the same location in the cache.

4. Improving Cache Performance • Compulsory misses can be reduced by increasing the block size • This uses spatial locality to preload data near past memory references

5. Block size example 16 Byte Block int A[1024]; for (i = 0; i < 1024; i++) A[i] = i; A[0] A[1] A[2] A[3] Miss, hit, hit, hit, <repeat> 25% miss rate let A be at address 0x7000 0000 Memory accesses: W 0x7000 0000 4 W 0x7000 0004 4 W 0x7000 0008 4 W 0x7000 000C 4 W 0x7000 0010 4 W 0x7000 0014 4 32 Byte Block A[0] A[1] A[2] A[3] A[4] A[5] A[6] A[7] Miss, hit, hit, hit, hit, hit, hit, hit, <repeat> 12.5% miss rate

6. Improving Cache Performance • Capacity misses are related to the number of indexes in the cache • Increasing cache size, increasing the number of indexes will reduce capacity misses • Decreasing block size also increases the number of cache indexes and can reduce capacity misses

7. Improving Cache Performance • Conflict misses can be reduced by changing the cache algorithm • A direct mapped cache has a many to one mapping of address to cache index • We want alternatives

8. Fully Associative Cache • Locate blocks anywhere in the cache • cache does not have an index • eliminates conflict misses • To check for hit, compare every tag in the cache to the tag for the memory address • search the cache - O(N)

9. Set Associative Cache • Fully associative is not practical for large caches • Compromise and allow cache blocks to be located in any of N indexes (called ways) which leads to: • N-way set associative cache

10. 2-way Associative • Two equally-sized DM caches • An address can be stored in the first or second DM cache • using the same index • Compare tags to check

11. Indexes in an N-way Cache

12. Example of a 4-way cache address Assume a 4K-byte cache, 1K words, N = 4 (4-way cache) Block size of 64 byte, or 8 words (1 way is 1K-byte, has 16 blocks) cache_index_range = cache_size / (4*block_size) = 4096/(4*64) = 16 (range 0-15) Assume an memory address of 103210 then 4-way cache index = (1032/256) % 16 = 4 % 16 = 4 (4*64 == 256) 4-way cache block offset = 1032 % 64 = 8 This piece of data can be in any of the four cache “ways”, 0, 1, 2, or 3. Need another search to identify it by comparing tag.

13. Associative Cache LRU Replacement • Add a saturating counter (age) • on a cache hit, set the counter to zero and increment the other way’s counter(s). • When something new needs to be cached, replace the way where the count is largest (least recently used) If there is a tie, choose randomly.

14. 4-way Set Associative MIPS Cache What is the block size? • 1 Byte • 2 Bytes • 4 Bytes • 8 Bytes • 16 Bytes

15. 4-way Set Associative MIPS Cache What is the block size? • 1 Byte • 2 Bytes • 4 Bytes • 8 Bytes • 16 Bytes

16. 4-way Set Associative MIPS Cache Explanation: The block size is 4 bytes because we used 2 bits to represent the block size (the last 2 bits of the adress.)

17. 4-way Set Associative MIPS Cache What is the cache size? • 512 Byte • 1 KB • 2 KB • 4 KB • 8 KB

18. 4-way Set Associative MIPS Cache What is the cache size? • 512 Byte • 1 KB • 2 KB • 4 KB • 8 KB

19. 4-way Set Associative MIPS Cache Explanation: Each cache unit has 256 blocks, each block is 4 bytes, thus each cache unit has 256*4 = 1024 bytes, or 1 KB. With four cache units, the total size is 4 KB.

20. 4-way Set Associative MIPS Cache What is the index for this address 0x0040F00C ? • 0x00 • 0x03 • 0x40 • 0x0C • 0xf00C

21. 4-way Set Associative MIPS Cache What is the index for this address 0x0040F00C ? • 0x00 • 0x03 • 0x40 • 0x0C • 0xf00C

22. 4-way Set Associative MIPS Cache Explanation: The cache index for address 0x0040F00C(or any address) is the right-most 10 bits, 8 bit for block index within the cache and 2 bits for the byte address within the block. So technically the cache index for this address should be 00 0000 1100, or 0x0C.

23. 4-way Set Associative MIPS Cache How large would the cache be if we increase the block size to 16 Bytes while keeping the number of indexes the same? • 1 KB • 2 KB • 4 KB • 8 KB • 16 KB

24. 4-way Set Associative MIPS Cache How large would the cache be if we increase the block size to 16 Bytes while keeping the number of indexes the same? • 1 KB • 2 KB • 4 KB • 8 KB • 16 KB

25. 4-way Set Associative MIPS Cache Explanation: if we keep the number of indexes the same, i.e., 256 and increase the block size to 16 bytes from the original 4 bytes, then each cache unit will have the size of 256*16 = 16 KB. Note that in this case, the number of bits needed to represent the cache address will increase to 12 bits, thus the bits devoted to tags will be 10 bits.

26. 4-way Set Associative MIPS Cache How many bits are needed for the tag if we increase the block size to 16 Bytes while keeping the number of indexes the same? • 20 • 21 • 22 • 23 • 24

27. 4-way Set Associative MIPS Cache How many bits are needed for the tag if we increase the block size to 16 Bytes while keeping the number of indexes the same? • 20 • 21 • 22 • 23 • 24

28. Conflicting Cache Requirements • Want low miss rate • large size (low capacity miss) • large blocks (low compulsory miss) • high associativity (low conflict miss) • We want the hit time to be fast • small size (indexing memory takes time) • lower associativity (reduce tag searching)