1 / 40

Improving Memory Access: the Cache

EECS 322 Computer Architecture. Improving Memory Access: the Cache. Instructor: Francis G. Wolff wolff@eecs.cwru.edu Case Western Reserve University This presentation uses powerpoint animation: please viewshow. Review: Models.

wyatt-blankenship
Download Presentation

Improving Memory Access: the Cache

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EECS 322 Computer Architecture Improving Memory Access: the Cache Instructor: Francis G. Wolff wolff@eecs.cwru.edu Case Western Reserve University This presentation uses powerpoint animation: please viewshow

  2. Review: Models Single-cycle model(non-overlapping)• The instruction latency executes in a single cycle• Every instruction and clock-cycle must bestretched to the slowest instruction (p.438) Multi-cycle model (non-overlapping) • The instruction latency executes in multiple-cycles • The clock-cycle must be stretched to the slowest step • Ability to share functional units within the execution of a single instruction Pipeline model (overlapping, p. 522)• The instruction latency executes in multiple-cycles • The clock-cycle must be stretched to the slowest step• The throughput is mainly one clock-cycle/instruction• Gains efficiency by overlapping the execution of multiple instructions, increasing hardware utilization. (p. 377)

  3. Review: Pipeline Hazards Pipeline hazards • Solution #1 always works (for non-realtime) applications: stall. Structural Hazards (i.e. fetching same memory bank) • Solution #2: partition architecture Control Hazards (i.e. branching) • Solution #1: stall! but decreases throughput • Solution #2: guess and back-track • Solution #3: delayed decision: delay branch & fill slot Data Hazards (i.e. register dependencies) • Worst case situation • Solution #2: re-order instructions • Solution #3: forwarding or bypassing: delayed load

  4. Review: Single-Cycle Datapath 2 adders: PC+4 adder, Branch/Jump offset adder I n s t r u c t i o n Harvard Architecture: Separate instruction and data memory And M A d d u x Add Result 4 Branch RegWrite S h i f t l e f t 2 MemWrite MemRead RegDst ALUctl 3 R e a d ALUSrc MemtoReg R e a d r e g i s t e r 1 P C R e a d a d d r e s s R e a d d a t a 1 Z e r o r e g i s t e r 2 A L U A L U R e a d W r i t e R e a d M A d d r e s s r e s u l t M u d a t a r e g i s t e r d a t a 2 M u I n s t r u c t i o n x u x W r i t e m e m o r y D a t a x d a t a m e m o r y W r i t e d a t a 3 2 1 6 S i g n e x t e n d

  5. Review: Multi vs. Single-cycle Processor Datapath M e m t o R e g A L U S r c B A L U O p Combine adders: add 1½ Mux & 3 temp. registers, A, B, ALUOut Combine Memory: add 1 Mux & 2 temp. registers, IR, MDR I o r D M e m R e a d M e m W r i t e I R W r i t e R e g D s t R e g W r i t e A L U S r c A P C 0 0 I n s t r u c t i o n R e a d M M [ 2 5 – 2 1 ] r e g i s t e r 1 A d d r e s s u u x R e a d x A I n s t r u c t i o n R e a d Z e r o d a t a 1 M e m o r y 1 1 [ 2 0 – 1 6 ] r e g i s t e r 2 A L U A L U A L U O u t 0 M e m D a t a R e g i s t e r s r e s u l t I n s t r u c t i o n W r i t e R e a d M B [ 1 5 – 0 ] r e g i s t e r d a t a 2 0 u I n s t r u c t i o n W r i t e x M [ 1 5 – 1 1 ] I n s t r u c t i o n 4 1 W r i t e d a t a 1 u r e g i s t e r d a t a 2 x 0 I n s t r u c t i o n 3 [ 1 5 – 0 ] M u x M e m o r y 1 1 6 3 2 d a t a A L U S h i f t S i g n r e g i s t e r c o n t r o l l e f t 2 e x t e n d I n s t r u c t i o n [ 5 – 0 ] Single-cycle= 1 ALU + 2 Mem + 4 Muxes + 2 adders + OpcodeDecoders Multi-cycle = 1 ALU + 1 Mem + 5½ Muxes + 5 Reg (IR,A,B,MDR,ALUOut) + FSM

  6. Review: Multi-cycle Processor Datapath M e m t o R e g A L U S r c B A L U O p Single-cycle= 1 ALU + 2 Mem + 4 Muxes + 2 adders + OpcodeDecoders Multi-cycle = 1 ALU + 1 Mem + 5½ Muxes + 5 Reg (IR,A,B,MDR,ALUOut) + FSM I o r D M e m R e a d M e m W r i t e I R W r i t e R e g D s t R e g W r i t e A L U S r c A P C 0 0 I n s t r u c t i o n R e a d M M [ 2 5 – 2 1 ] r e g i s t e r 1 A d d r e s s u u x R e a d x A I n s t r u c t i o n R e a d Z e r o d a t a 1 M e m o r y 1 1 [ 2 0 – 1 6 ] r e g i s t e r 2 A L U A L U A L U O u t 0 M e m D a t a R e g i s t e r s r e s u l t I n s t r u c t i o n W r i t e R e a d M B [ 1 5 – 0 ] r e g i s t e r d a t a 2 0 u I n s t r u c t i o n W r i t e x M [ 1 5 – 1 1 ] I n s t r u c t i o n 4 1 W r i t e d a t a 1 u r e g i s t e r d a t a 2 x 0 I n s t r u c t i o n 3 [ 1 5 – 0 ] M u x M e m o r y 1 1 6 3 2 d a t a A L U S h i f t S i g n r e g i s t e r c o n t r o l l e f t 2 e x t e n d I n s t r u c t i o n [ 5 – 0 ] 5x32 = 160 additional FFs for multi-cycle processor over single-cycle processor

  7. Figure 6.25 2 W3 M4 EX 2 W3 M PC 32 bits PC 32 2 W PC32 M D R 32 Z 1 A32 PC 32 bits IR 32 bits B32 ALUOut32 ALUOut32 Datapath Registers Si32 B32 160 FFs D5 RT5 D5 + 213 FFs RD5 + 16 FFs 213+16 = 229 additional FFs for pipeline over multi-cycle processor

  8. Review: Overhead Single-cycle model• 8 ns Clock (125 MHz), (non-overlapping)• 1 ALU + 2 adders • 0 Muxes• 0 Datapath Register bits (Flip-Flops) Chip Area Speed Multi-cycle model • 2 ns Clock (500 MHz), (non-overlapping)• 1 ALU + Controller • 5 Muxes• 160 Datapath Register bits (Flip-Flops) Pipeline model • 2 ns Clock (500 MHz), (overlapping) • 2 ALU + Controller • 4 Muxes• 373 Datapath + 16 Controlpath Register bits (Flip-Flops)

  9. Review: Data Dependencies: no forwarding 8 Clock 1 2 3 4 5 6 7 WB IF ID EX M WB IF ID ID ID EX M Write1st Half Read2nd Half Stall Stall sub $2,$1,$3 and $12,$2,$5 Suppose every instruction is dependant = 1 + 2 stalls = 3 clocks MIPS = Clock = 500 Mhz = 167 MIPS CPI 3

  10. Review: R-Format Data Dependencies: Hazard Conditions 1a Data Hazard (2 stalls): EX/MEM.$rd = ID/EX.$rs sub $2, $1, $3 sub $rd, $rs, $rt and $12, $2, $5 and $rd, $rs, $rt 1b Data Hazard (2 stalls): EX/MEM.$rd = ID/EX.$rt sub $2, $1, $3 sub $rd, $rs, $rt and $12, $1, $2 and $rd, $rs, $rt 2a Data Hazard (1 stall): MEM/WB.$rd = ID/EX.$rs sub $2, $1, $3 sub $rd, $rs, $rt and $12, $1, $5 sub $rd, $rs, $rt or $13, $2, $1 and $rd, $rs, $rt 2b Data Hazard (1 stall): MEM/WB.$rd = ID/EX.$rt sub $2, $1, $3 sub $rd, $rs, $rt and $12, $1, $5 sub $rd, $rs, $rt or $13, $6, $2 and $rd, $rs, $rt

  11. Clock 1 2 3 4 5 6 WB IF ID EX M WB IF ID EX M Can R-Format dependencies completely be resolved by forwarding? and $12,$2,$5beq $12,$0,L7 Data Dependencies (hazard 1a and 1b): with forwarding sub $2,$1,$3 and $12,$2,$5 DetectedData Hazard 1a ID/EX.$rs = EX/M.$rd

  12. Load Data Hazards: Hazard detection unit (page 490) Stall Example lw $2, 20($1) lw $rt, addr($rs) and $4, $2,$5 and $rd, $rs,$rt Stall Condition Source Destination } IF/ID.$rsIF/ID.$rt = ID/EX.$rt  ID/EX.MemRead=1 No Stall Example: (only need to look at next instruction) lw $2, 20($1) lw $rt, addr($rs) and $4, $1, $5 and $rd, $rs, $rt or $8, $2, $6 or $rd, $rs, $rt

  13. Clock 1 2 IF ID IF Load Data Dependencies: with forwarding 3 4 5 6 WB lw $2,20($1) EX M and $4,$2,$5 WB ID ID EX M DetectedData Hazard IF/ID.$rs = EX/M.$rt Load dependencies cannot be completely resolved by forwarding Even through the Load stalls the next instruction, the stall time is added to the load instruction and not the next instruction. Load time = 1 (no dependancy) to 2 (with dependency on next instruction)

  14. Delay slot Before After add $4,$6,$6 beq $1,$3,7 beq $1,$3,L7 add $4,$6,$6 • • • • • • L7: lw $4, 50($7) L7: lw $4, 50($7) Can you move the add instruction into the delay slot? No - but a delay slot still requires an instruction add $4,$6,$6 add $4,$6,$6 beq $1,$4,L7 beq $1,$4,L7 add $0,$0,$0

  15. Branch Hazards: Soln #3, Delayed Decision 8 Clock 1 2 3 4 5 6 7 WB beq $1,$3,7 IF ID EX M instruction was before the branch add $4,$6,$6 WB IF ID EX M Do not need to discard instruction lw $4, 50($7) WB IF ID EX M Decision made in ID stage: branch

  16. Summary: Instruction Hazards No-ForwardingForwardingHazard R-Format 1-3 1 Data Load 1-3 1-2 Data, Structural Store 1 1-2 Structural No Delay SlotDelay SlotHazard Branch 2 1 Control(decision is made in the ID stage) Branch 3 1 Control(decision is made in the EX stage) Jump 2 1 Structural Hazard: Instruction & Data memory combined.

  17. Performance, page 504 Also known as the instruction latency with in a pipeline Pipeline throughput Instruction Single-Cycle Multi-CycleClocks PipelineCycles InstructionMix loads 1 5 1.5(50% dependancy) 23% stores 1 4 1 13% arithmetic 1 4 1 43% branches 1 3 1.25(25% dependancy) 19% jumps 1 3 2 2% Clockspeed 125 Mhz8 ns 500 Mhz2 ns 500 Mhz2 ns CPI 1 4.02 1.18 =  Cycles*Mix MIPS 125 MIPS 125 MIPS 424 MIPS = Clock/CPI load instruction time = 50%*(1 clock) + 50%*(2 clocks)=1.5 branch time = 75%*(1 clocks) + 25%*(2 clocks) = 1.25

  18. Pipelining and the cache (Designing…,M.J.Quinn, ‘87) Instruction Pipelining is the use of pipelining to allow more than one instruction to be in some stage of execution at the same time. Ferranti ATLAS (1963): Pipelining reduced the average time per instruction by 375% Memory could not keep up with the CPU, needed a cache. Cache memory is a small, fast memory unit used as a buffer between a processor and primary memory

  19. Principle of Locality • Principle of Localitystates that programs access a relatively small portion of their address space at any instance of time • Two types of locality • Temporal locality (locality in time)If an item is referenced, thenthe same item will tend to be referenced soon“the tendency to reuse recently accessed data items” • Spatial locality (locality in space)If an item is referenced, thennearby items will be referenced soon“the tendency to reference nearby data items”

  20. Memories Technology and Principle of Locality Memory Technology Typical access time $ per Mbyte SRAM 5-25 ns $100-$250 DRAM 60-120 ns $5-$10 Magnetic Disk 10-20 million ns $0.10-$0.20 • Faster Memories are more expensive per bit • Slower Memories are usually smaller in area size per bit

  21. Cheaper Cost $$$ More Capacity Faster Memory Hierarchy CPU Registers Pipelining Cache memory Primary real memory Virtual memory (Disk, swapping)

  22. Basic Cache System

  23. Cache Terminology A hit if the data requested by the CPU is in the upper level Hit rate or Hit ratio is the fraction of accesses found in the upper level Hit time is the time required to access data in the upper level = <detection time for hit or miss> + <hit access time> A miss if the data is not found in the upper level Miss rate or (1 – hit rate) is the fraction of accesses not found in the upper level Miss penalty is the time required to access data in the lower level = <lower access time>+<reload processor time>

  24. Figure 7.2 P r o c e s s o r Time 3: deliver to CPU D a t a a r e t r a n s f e r r e d Time 2: fetch from lower level into cache Cache Example Time 1: Hit: in cache Time 1: Miss Hit time = Time 1 Miss penalty = Time 2 + Time 3

  25. Cache Memory Technology: SRAM see page B-27 • Why use SRAM (Static Random Access Memory)? • Speed. The primary advantage of an SRAM over DRAM is speed. The fastest DRAMs on the market still require 5 to 10 processor clock cycles to access the first bit of data. SRAMs can operate at processor speeds of 250 MHz and beyond, with access and cycle times equal to the clock cycle used by the microprocessor • Density. when 64 Mb DRAMs are rolling off the production lines, the largest SRAMs are expected to be only 16 Mb. see reference: http://www.chips.ibm.com/products/memory/sramoperations/sramop.html

  26. Cache Memory Technology: SRAM (con’t) • Volatility. Unlike DRAMs, SRAM cells do not need to be refreshed. SRAMs are available 100% of the time for reading & writing. • Cost. If cost is the primary factor in a memory design, then DRAMs win hands down. If, on the other hand, performance is a critical factor, then a well-designed SRAM is an effective cost performance solution.

  27. Cache Memory Technology: SRAM Block diagram

  28. Cache Memory Technology: SRAM timing diagram

  29. Cache Memory Technology: SRAM 1 bit cell layout

  30. Ref: http://www.msm.cam.ac.uk/dmg/teaching/m101999/Ch8/index.htm

  31. see page B-31

  32. Memory Technology: DRAM Evolution

  33. Direct Mapped Cache • Direct Mapped: assign the cache location based on the address of the word in memory • cache_address = memory_address modulo cache_size; Observe there is a Many-to-1 memory to cache relationship

  34. Direct Mapped Cache: Data Structure There is a Many-to-1 relationship between memory and cache How do we know whether the data in the cache corresponds to the requested word? tags • contain the address information required to identify whether a word in the cache corresponds to the requested word. • tags need only to contain the upper portion of the memory address (often referred to as a page address) valid bit • indicates whether an entry contains a valid address

  35. Direct Mapped Cache: Temporal Example Figure 7.6 100 111 110 Index 101 000 011 001 010 N N N N Valid N N N N Tag Data Y Y 11 10 Memory[10110] Memory[11010] Miss: valid lw $1,10 110 ($0) lw $1,22($0) lw $2,11 010 ($0) Miss: valid lw $2,26($0) lw $3,10 110 ($0) Hit! lw $3,22($0)

  36. Direct Mapped Cache: Worst case, always miss! Figure 7.6 111 110 101 011 010 001 000 100 Index N N N N N N Valid N N Tag Data Y Y Y 10 00 11 Memory[10110] Memory[11110] Memory[00110] Miss: valid lw $1,10 110 ($0) lw $1,22($0) lw $2,11 110 ($0) Miss: tag lw $2,30($0) lw $3,00 110 ($0) Miss: tag lw $3,6($0)

  37. Direct Mapped Cache: Mips Architecture Figure 7.7 Tag Index Data Hit Compare Tags

  38. Modern Systems: Pentium Pro and PowerPC

More Related