**"Memory Performance: Understanding Hit Rate and Miss Penalty"**

2. Exam 2 Review • Two’s Complement Arithmetic • Ripple carry ALU logic and performance • Look-ahead techniques, performance and equations • Basic multiplication and division ( non- restoring) algorithms • IEEE 754 floating point standard (definition provided) • Write a sequence of register transfers to implement a given instruction for MIPS • Given a set of Register Transfers, design the control needed for some component

3. Five Components of Computers Memory Control Input Datapath Output Processor Why is Memory Performance critical ?

4. Memory can’t be too fast or too large! • The desire of every programmer is to have • UNLIMITED HIGH SPEED MEMORY • The goal of the designer is to create the illusion • of UNLIMITED HIGH SPEED MEMORY • that is affordable

5. High Speed means 1 clock cycle SRAM – Static Random Access Memory Flip – Flops 4 to 6 transistors per bit Access time in few nsec About $5000 per GByte

6. High Speed means 1 clock cycle SRAM – Static Random Access Memory Flip – Flops 4 to 6 transistors per bit Access time in few nsec About $5000 per GByte DRAM – Dynamic Random Access Memory Charged Capacitor ( Refresh Required) 1 transistor per bit Access time 50 to 100 nsec Less than $75 per GByte

7. High Speed means 1 clock cycle SRAM – Static Random Access Memory Flip – Flops 4 to 6 transistors per bit Access time in few nsec About $5000 per GByte DRAM – Dynamic Random Access Memory Charged Capacitor ( Refresh Required) 1 transistor per bit Access time 50 to 100 nsec Less than $75 per GByte Technology continues to improve, but the ratios stay about the same

8. Memory Hierarchy Processor Transfer in 1 clock cycle Cache Memory Transfer > 10 clock cycles Main Memory

9. Memory Hierarchy Processor Objective is to have a copy of the data in the cache that the processor needs most of the time. Transfer in 1 clock cycle Cache Memory Transfer > 10 clock cycles Main Memory

10. Hit is a memory access that is found in the cache • Miss is a memory access that is not found in the cache

11. Hit is a memory access that is found in the cache • Miss is a memory access that is not found in the cache • Hit Rate and Miss Rate are the fraction of Hits or Misses • of all memory accesses for a program(s) • Note: Hit Rate + Miss Rate = 1

12. Hit Time is the time to access the cache memory, including the time to determine a Hit or Miss

13. Hit Time is the time to access the cache memory, • including the time to determine a Hit or Miss • Miss Penalty is the total timeto complete the memory • transaction for a miss. • For a read, it is the time to transfer the data from the • main memory to the cache and the time for the • processor to access the cache. [Reads Dominate]

14. Hit Time is the time to access the cache memory, • including the time to determine a Hit or Miss • Miss Penalty is the total timeto complete the memory • transaction for a miss. • For a read, it is the time to transfer the data from the • main memory to the cache and the time for the • processor to access the cache. [Reads Dominate] • Miss Time = Hit Time + Miss Penalty is the total time to • complete the memory transaction for a miss including • the time to determine a miss.

15. Memory Performance Measure Average memory access time ( in seconds or clock cycles) = Hit Rate * Hit time + Miss Rate * Miss Time

16. Memory Performance Measure Average memory access time ( in seconds or clock cycles) = Hit Rate * Hit time + Miss Rate * Miss Time = (1 – Miss Rate) * Hit Time + Miss Rate * ( Hit time + Miss Penalty)

17. Memory Performance Measure Average memory access time ( in seconds or clock cycles) = Hit Rate * Hit time + Miss Rate * Miss Time = (1 – Miss Rate) * Hit Time + Miss Rate * ( Hit time + Miss Penalty) = Hit Time + Miss Rate * Miss Penalty

18. Memory Performance Measure Average memory access time ( in seconds or clock cycles) = Hit Rate * Hit time + Miss Rate * Miss Time = (1 – Miss Rate) * Hit Time + Miss Rate * ( Hit time + Miss Penalty) = Hit Time + Miss Rate * Miss Penalty Ex: Hit Time is 1 clock cycle Miss Penalty is 20 clock cycles Ave Access = 1 + 20 *Miss Rate

19. Memory Performance Measure Average memory access time ( in seconds or clock cycles) = Hit Rate * Hit time + Miss Rate * Miss Time = (1 – Miss Rate) * Hit Time + Miss Rate * ( Hit time + Miss Penalty) = Hit Time + Miss Rate * Miss Penalty Ex: Hit Time is 1 clock cycle Miss Penalty is 20 clock cycles Ave Access = 1 + 20 *Miss Rate 1.2 1% 1.6 3% 2.0 5%

20. Memory Performance Measure Average memory access time ( in seconds or clock cycles) = Hit Time * Hit time + Miss Rate * Miss Time = (1 – Miss Rate) * Hit Time + Miss Rate * ( Hit time + Miss Penalty) = Hit Time + Miss Rate * Miss Penalty Ex: Hit Time is 1 clock cycle Miss Penalty is 20 clock cycles 10 clock cycles Ave Access = 1 + 20 *Miss Rate Ave Access 1.2 1% 1.1 1.6 3% 1.3 2.0 5% 1.5

21. How can we get a Low Miss Rate < 10% ? Principle of Locality for programs Temporal locality ( locality in time) If a memory address is accessed, it will tend to be accessed again soon.

22. How can we get a Low Miss Rate < 10% ? Principle of Locality for programs Temporal locality ( locality in time) If a memory address is accessed, it will tend to be accessed again soon. Spatial locality ( locality in space) If a memory address is accessed, addresses that are close by will tend to be referenced soon

23. How can we get a Low Miss Rate < 10% ? Principle of Locality for programs Temporal locality ( locality in time) If a memory address is accessed, it will tend to be accessed again soon. Spatial locality ( locality in space) If a memory address is accessed, addresses that are close by will tend to be referenced soon 90/10 Rule: A program spends 90% of its execution in only 10% of the code.

24. How can we get a Low Miss Rate < 10% ? Principle of Locality for programs Temporal locality ( locality in time) If a memory address is accessed, it will tend to be accessed again soon. Spatial locality ( locality in space) If a memory address is accessed, addresses that are close by will tend to be referenced soon 90/10 Rule: A program spends 90% of its execution in only 10% of the code. Predict the memory locations needed with +90% accuracy