1 / 39

Continuing the Discussion of Memory Hierarchy

Continuing the Discussion of Memory Hierarchy. Memory Secondary Storage Disk. Memory Main Memory DRAM. Memory Level 2 Cache SRAM. Memory Level 1 Cache. Memory registers. Topics of this lecture set. Virtual Memory. Illusion of Memory Size Much Greater Than Physical Memory Size

oliver-gill
Download Presentation

Continuing the Discussion of Memory Hierarchy

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Continuing the Discussion of Memory Hierarchy Memory Secondary Storage Disk Memory Main Memory DRAM Memory Level 2 Cache SRAM Memory Level 1 Cache Memory registers Topics of this lecture set

  2. Virtual Memory • Illusion of Memory Size Much Greater Than Physical Memory Size • Exploits Memory Hierarchy to Keep Average Access Time Low • At Least 2 Storage Levels: Main and Secondary • Virtual Address — Address Used By the Programmer • Memory Address — Address in Physical Memory (Main Memory) (Secondary Storage)

  3. Secondary Storage cache Main memory Cache Design Virtual Memory Design Similarities Between Cache and Virtual Memory • Both Use Two Levels of Memories • Higher Level: Faster and Smaller • Lower Level: Slower and Larger • Both Rely on the Principle of Locality • Both Use Associativity to Reduce Conflicts • Both Need to Decide Which Block in Higher Level has to be Replaced Upon Miss

  4. Differences Between Cache and Virtual Memory • Cache is several orders of magnitude faster than virtual memory, while virtual memory is several orders of magnitude larger than cache • Consequently • Virtual memory can use software to track blocks in use while cache has to use hardware • The cost to implement full associativity is low for Virtual memory and very high for cache • Virtual memory can use more sophisticated block replacement algorithms • Virtual memory has to use write-back while cache can use write-back or write-through

  5. Issues in Virtual Memory Design • Mapping — How to Map the Blocks in Virtual Memory (i.e., the Secondary Storage) to the Main Memory • Block Placement — How to find a Block in the virtual memory and where to put it in the Main Memory (A consequence of full associativity) • Fetch Policy — How to Know When It Is Necessary to Look for the Needed Data From Virtual Memory (You would try to avoid fetching the virtual memory unless it is absolutely necessary due to the slow access time) • Block Replacement — How to Free up a Region in Main Memory to Place a New Block from Virtual Memory • Write Policy — What to Do If a Block in Main Memory Has to Be Replaced

  6. The Basic Unit of Transferring Data From Secondary Storage to Main Memory • Equal Size Blocks • Paging: Block of Fixed Size in Virtual Memory • Page Frame: Physical Address Space for Each Page Example: Alpha has 8KB, 64KB, 512KB, 4MB Page Sizes • Variable Sized Blocks • Segmentation Example: Burroughs B4500 • Multiple Page Blocks • Segments Consist of Multiple Pages Note: Paging is used in most modern machines due to its simplicity

  7. Example Paging Organization Page Size = 1 Kbytes Address Translation Map Smaller (Less Pages) Larger (More Pages)

  8. Page Table • To keep track which page of the secondary store is in main memory and translate virtual memory address into physical address • The page table itself is stored in the main memory • Each entry has a valid bit, access right, and the page’s physical address • The higher order bits of a virtual memory address (number of bits depending on page size) is used to index the entries. This is how the virtual memory address is translated into physical address • Usually each program has its own page table. The page table of the running program is pointed to by a Page Table Pointer Register 32 bits 1 bits 1 bits 24 bits Virtual Page # Valid Access Right Physical Page Address Page Table Pointer Note 00001000 0 R/W 29B000 00002000 1 R 737000 ... ... ... ... 000F000 1 X C37000 Note: This register and the page table along with the program counter constitute the state of a program, also known as a process. The page table is loaded from disk to memory when the process is active. Operating System has to make sure no page is assigned to two active programs.

  9. Page Table Example Main (Physical) Memory Disk Page Table of Program A Data of Virtual Page 1010 V R/W Virtual Page # Physical Page # Data of Virtual Page 1011 1010 3 1 r/w Data of Virtual Page 1012 1011 0 r/w Data of Virtual Page 1013 1012 0 - Data of Virtual Page 1014 1013 1 1 X Data of Virtual Page 1015 1014 0 r Data of Virtual Page 1016 1015 0 - Data of Virtual Page 1017 1016 0 - Data of Virtual Page 1018 1017 0 1 r/w 1018 2 1 r Data Area of Program A Physical Page # Data of Virtual Page 1017 0 Data of Virtual Page 1013 1 Access Rights: R = Read- Only R/W = Read/Write X = Execute Only Data of Virtual Page 1018 2 Data of Virtual Page 1010 3

  10. Protection of Pages by Page Frame Table • The Page Frame Table is Used by the operating system to know how the pages in main memory are allocated to different active jobs, and to provide the information to use in deciding what pages to replace. • Some frequently used pages such as critical OS functions can be “locked” into areas of main memory Set if page is accessed User program name Set if page is modified

  11. Read Address Write Address V=1 V=1 V=0 Old Page new phy addr New Page To Cache To Cache Address Mapping • Address Translation Determines If Main Memory Has the Requested Page by Examining the Valid Bit of the Page in the Page Table • If the Requested Page Is Not in Main Memory, Operating System Transfers Data from Secondary Memory to Main Memory and Then Set the Valid Bit. Write the old page back to memory if necessary (e.g., page modified but not saved).

  12. 20 18 Physical Address Access Rights Physical Page # To Memory if V=1 Example: Address Mapping with 1KB Pages • Page Table Located in Physical Memory • V = Valid Bit: • V = 1: Page is in Main Memory • Access Rights: R = Read- Only, R/ W = Read/ Write, X = Execute Only

  13. Program X Unused Program X Program X Program Y Program Y Program X memory Wasted Memories memory Advantages and Disadvantages of Paging • Advantages • Fixed page boundary (usually address = 2N) significantly simplifies control logic (e.g., page number is just the upper bits of the address) • Flexible: Pages of the same program do not have to be contiguous. Use the page table to locate individual pages • Disadvantages • Fragmentation: Disjointed Areas of Unused Memory Space • Internal Fragmentation: Program or Data Does not use an Integral Number of Pages so Part of the Last Page is Wasted: Page Table for Program X Page 1 Page 2 Page 3 Page 4 memory

  14. Program Y Segment pointer Program X data Off-set Boundary Reg memory Segmentation and Paged Segments • Segmentation tries to remove the fragmentation by using vaiable block size, but it requires much more complex control logic (a segment pointer, an off-set pointer, and a boundary register) • Paged Segments • Programs and Data Treated as Logical Entities (i.e., Segments of Differing Size) • Each Segment Is then Broken into Fixed Sized Pages and Has Its Own Page Table • Use of Segments is Invisible to the User • Allows Protection to be Made on the Basis of Logical Entities (Bits Programs Set in Segment Table) • More Complex Address Translation Page table for segment

  15. Trade-Offs Of Page Size • Large Page Size Increases Internal Fragmentation, but Decreases the Number of Pages and Page Table Size and Thus Simplifies the Management and Control of the Pages • Small Page Size Decreases Internal Fragmentation, but Increases Page Table Size and Complicates the Management and Control of the Pages • Trend Toward Large Page Sizes • Large Memories as RAM Costs Drop • Wider Gap Between Processor Speed and Disk Speed • Programmers Want Larger Virtual Address Space • 4KB Pages are Common

  16. V Main Memory Phy Page # Virtual Page # in Virtual Address 0 3 1 1 4 1 2 0 1 3 1 5 1 4 0 5 0 0 1 3 1 Disk Page Table of Program Placing a Block in Main Memory • Since the secondary storage is so much slower than the CPU and main memory, software can be used to manage block placement in main memory without noticeable performance impact • Now, it is practical to use full associativity for block replacement to reduce page conflicts. • The full associativity is managed by the page table (as oppose to using tags and comparators in cache). The virtual page number serves as the “index”, and no “tag” is necessary because the index is a full address • Each program can have its own page table and a hardware page table register is used to point to the active page table.

  17. Page Table for Large Address Space Two-level Page Tables 1K PTEs 4KB 32-bit address: 10 10 12 P1 index P2 index page offest 4 bytes • Assuming 4 KB pages, a single-level page table, 4 GB virtual address space requires 1 M page table entries stored in memory • Assuming 4 KB pages, a two-level page table, 4 GB virtual address space requires 1 K entries in 1st level page table, 1000 2nd level page page of 1 K entries each. However, it is not necessary to load all 1000 2nd level page table in memory at the same time. 4 bytes

  18. Finding Data in Main Memory • Using page table to find a data in memory has a problem; every memory access requires a 2-step process • Step 1: Read page table in order to obtain a physical address • Step 2: Use the physical address to get the data • The 2-step process doubles the miss panelty of cache misses • Why not cache the most recently used portion of the page table also? This cache is called the Table Look-Aside Buffer (TLB) Tag Phy Page # V 1 3 1 Virtual Page # in Virtual Addr 0 2 1 4 5 1 0 TLB V Main Memory Phy Page # 0 2 0 1 1 4 1 1 2 2 0 1 3 1 3 5 1 4 4 0 5 5 0 6 0 1 7 3 1 8 Disk Page Table

  19. Translation Lookaside Buffer • Cache of Recently Used Page Table Entries • Can Be Fully Associative, Set Associative, or Direct Mapped • Direct Mapped TLB Example: index Note: Dirty bit indicates if the page in memory has been modified. If it has not been modified, it will be replaced without copying back to memory.

  20. Translation with a TLB (get physical address from page table in memory) See Class Example

  21. Virtual Address 31 12 11 0 Virtual Page Number Page Offset Valid Dirty Tag Physical Page # 12 20 TLB mux 20 = = = Physical Address = 2 14 Data Valid Tag Index Byte Offset Tag TLB Hit = 32 Cache Hit Data Virtual Memory and Cache MappingsExample: Decstation 3100 Note: Another important bookkeeping bit Write Access Bit for Write Protection Is Not Shown See Class Example

  22. Decstation 3100 TLB and Cache (Continued) Get Page Table Entry from Memory Get Data from Memory

  23. Virtual Memory Read Example Virtual Address Virtual Page Number Tag = 0111 Index = 01 Page Offset =10 TLB index Valid Dirty Tag Physical Page # 00 1 1 0111 011 TLB Miss! Get physical page # from memory 01 1 0 0110 111 10 1 0 1111 100 11 1 1 1111 101 mux Physical Address = = = = Tag Index Index Valid Tag Data 1 00 001 AA 1 01 010 FF 1 10 010 EE 1 11 001 BB TLB Hit Data Cache = Cache Hit Data

  24. 1 0 1 1 1 1 1 1 Virtual Memory Read Example Virtual Page # = 011101 Main (Physical) Memory Program A Page Table Page Frame Table V Page # P Page # P Page # V Page # R/W Dirty Prog V Used 011000 - 011010 0 r/w 000 1 A 011001 000010 111 001 1 r/w 0 B 011010 000011 000 010 1 - 1 B 011011 - 011100 011 0 X 0 A 011100 011110 1 r 011 100 0 A 011101 011111 0 - - 101 1 A Page not in physical memory! Need to get it from Disk! But where are you going to put it in the memory after you get the data from disk? 011110 000001 1 - 100 110 0 B 011111 011001 1 - 101 111 1 A Data Area of Program A Physical Page # Offset Data Physical Page # Offset Data 001 00 AA 010 00 11 01 00 01 FF 10 22 10 EE 11 BB 11 DD

  25. 1 0 1 1 1 1 1 1 Virtual Memory Read Example Virtual Page # = 011101 Main (Physical) Memory Program A Page Table Page Frame Table V Page # P Page # P Page # V Page # R/W Dirty Prog V Used 011000 - 011010 0 r/w 000 1 A 011001 000010 111 001 1 r/w 0 B 011010 000011 000 010 1 - 1 B 011011 - 011100 011 0 X 0 A 011100 011110 1 r 011 100 0 A 011101 011111 0 - - 101 1 A 011110 000001 1 - 100 110 0 B 011111 011001 1 - 101 111 1 A Data Area of Program A Least recently used Physical Page # Offset Data Physical Page # Offset Data 001 00 AA 010 00 11 01 00 01 FF 10 22 10 EE 11 BB 11 DD

  26. Assign Physical Page # 1 to Virtual Page 011101 001 r 1 AA 00 22 BB Virtual Memory Read Example Virtual Page # = 011101 Main (Physical) Memory Program A Page Table Page Frame Table V Page # P Page # P Page # V Page # R/W Dirty Prog V Used 011000 - 011010 0 r/w 000 1 1 A 011001 011101 111 001 1 r/w 1 0 A 011010 000011 000 010 1 - 1 1 B 011011 - 011100 011 0 X 1 0 A 011100 011110 1 r 011 100 1 0 A 011101 011111 0 - - 101 1 1 A 011110 000001 1 - 100 110 1 0 B 011111 011001 1 - 101 111 1 1 A Data Area of Program A Physical Page # Offset Data Physical Page # Offset Data 001 00 AA 010 00 11 01 00 01 FF 10 22 10 EE 11 BB 11 DD

  27. Virtual Memory Read Example Virtual Page # = 011101 Disk Offset Data Virtual Page # 00 80 011100 01 90 10 A0 11 B0 011101 00 C0 01 D0 10 E0 11 F0

  28. Update TLB with the Assigned Physical Page # 001 r 1 C0 AA D0 00 E0 22 BB F0 Virtual Memory Read Example Main (Physical) Memory Program A Page Table Page Frame Table V Page # P Page # P Page # V Page # R/W Dirty Prog V Used 011000 - 011010 0 r/w 000 1 1 A 011001 011101 111 001 1 r/w 1 0 A 011010 000011 000 010 1 - 1 1 B 011011 - 011100 011 0 X 1 0 A 011100 011110 1 r 011 100 1 0 A 011101 011111 0 - - 101 1 1 A 011110 000001 1 - 100 110 1 0 B 011111 011001 1 - 101 111 1 1 A Data Area of Program A Physical Page # Offset Data Physical Page # Offset Data 001 00 AA 010 00 11 01 00 01 FF 10 22 10 EE 11 BB 11 DD

  29. 1 0 0111 001 001 10 Virtual Memory Read Example Virtual Address Virtual Page Number Tag = 0111 Index = 01 Page Offset =10 TLB index Valid Dirty Tag Physical Page # 00 1 1 0111 011 TLB Updated and Physical Address can be obtained 01 1 0 0110 111 10 1 0 1111 100 11 1 1 1111 101 mux Physical Address = = = = Tag Index Index Valid Tag Data 1 00 001 AA 1 01 010 FF 1 10 010 EE 1 11 001 BB TLB Hit Data Cache = Cache Hit Data

  30. 001 10 Virtual Memory Read Example Virtual Address Virtual Page Number Tag = 0111 Index = 01 Page Offset =10 TLB index Valid Dirty Tag Physical Page # 00 1 1 0111 011 01 1 0 0111 001 10 1 0 1111 100 11 1 1 1111 101 mux Physical Address = = = = Tag Index Index Valid Tag Data 1 00 001 AA 1 01 010 FF Data Cache Miss! 1 10 010 EE 1 11 001 BB TLB Hit Data Cache = Cache Hit Data

  31. 001 r 1 C0 AA D0 00 22 E0 F0 BB Virtual Memory Read Example Main (Physical) Memory Program A Page Table Page Frame Table V Page # P Page # P Page # V Page # R/W Dirty Prog V Used 011000 - 011010 0 r/w 000 1 1 A 011001 011101 111 001 1 r/w 1 0 A 011010 000011 000 010 1 - 1 1 B 011011 - 011100 011 0 X 1 0 A 011100 011110 1 r 011 100 1 0 A 011101 011111 0 - - 101 1 1 A 011110 000001 1 - 100 110 1 0 B 011111 011001 1 - 101 111 1 1 A Data Area of Program A Physical Page # Offset Data Physical Page # Offset Data 001 00 AA 010 00 11 01 00 01 FF 10 22 10 EE 11 BB 11 DD

  32. 001 10 1 001 E0 Virtual Memory Read Example Virtual Address Virtual Page Number Tag = 0111 Index = 01 Page Offset =10 TLB index Valid Dirty Tag Physical Page # 00 1 1 0111 011 01 1 0 0111 001 10 1 0 1111 100 11 1 1 1111 101 mux Physical Address = = = = Tag Index Index Valid Tag Data 1 00 001 AA 1 01 010 FF Data Cache Updated! 1 10 010 EE 1 11 001 BB TLB Hit Data Cache = Cache Hit Data

  33. When to Fetch Secondary Storage? • Page Fault: • Page Not In Main Memory (i.e., Valid Bit = 0 on Access) • Current Process Suspended While a Page is Fetched From Secondary Storage (may switch to another process while waiting for data from disk) • Fetch Policies • Demand Paging • Pages Loaded Due to Page Faults Only • If the Page to be Replaced Has Been Written to (i. e., Dirty), It Must be Written to Disk Before the New Page is Fetched. • Prepaging • Anticipate References and Load Pages Before Their Use • Reduces Wait Time for a Page Transfer • Difficult to Predict Future References • Most Common Implementation is Demand Paging

  34. Page Replacement • Similar to Cache Block Replacement, Except that Software Can Be Used to Decide Which Block Should Be Replaced. Therefore, More Sophisticated Algorithms Can Be Used. • First-in-First-out • Replace the Page that has been in Memory the Longest • Replacement of Old but Frequently Used Page Can Happen • Easy to Implement • Least Recently Used • Replace the Least Recently Used Page • Need Access History, It Is Easier in This Case Because It Can Be Implemented in Software Due to the Long Latency of Disk Access. • Good Performance, Uses Principle of Locality • Page- Fault Frequency (PFF) • Dynamically Assign Pages to Jobs Based on their Page Fault Rate • Take Pages from Jobs that Seldom Fault and Give Pages to Jobs that Fault Frequently

  35. Page Replacement (Continued) • During a Page Miss: • A processor switches to another job whose pages are present • Returns to the current job after the missing pages have been fetched (Hundreds of thousands of cycles later) • Working Set = the number of pages a job needs to achieve a “Reasonably Low” Miss Ratio. • It varies from program to program, but with too few pages the job thrashes, and spends all of its time waiting for pages • Page allocation and replacement strategies should achieve good processing efficiency over the set of jobs in the machine • Too Many Jobs and the whole system thrashes • Too Few Pages for one job and that job gets nothing done

  36. Example Implementation of LRU in Virtual Memory • Associated with each page is a used bit such that • used bit = 1 if the page has been referenced in recent past • = 0 otherwise • If replacement is necessary, choose any page frame such that its used bit is 0. This is a page that has not been referenced in the recent past. • If more than one page has used bit set to 0, choose a page whose dirty bit is also 0. This page has not been changed and therefore no need to write back to disk. • Question: How to update the used bits?

  37. 0x1200 0x1000 Replace Replace 1 1 Read Write Read One Possible Way to Update the Used Bit Add a Last Replaced Pointer (LRP) in Page Fault Handler: • LRP points to the entry that have just been replaced • If replacement is to take place, advance LRP to next entry until one with a 0 bit is found; this is the target for replacement; As a side effect, all examined page table entries have their reference bits set to zero. Search for an unused entry, reset used bits of examined entries in the process Assume the Program needs to replace a page again Page Frame Table Page # used Virtual Address • Last Replaced Pointer 0 0x0200 0 0x0400 0 1 1 0 0x0600 1 0 2 0 0x0800 1 0 3 0 0x0A00 0 4 0x0C00 5 0 • Control scheme is simple • But this is only an approximation of LRU. No guarantee the replaced entries are the LEAST recently used

  38. Write Policy • Due to long latency of disk access, only write-back is practical for virtual memory. • If accessed page in memory is valid, processor can write data to the page at memory speed rather than at the disk speed, which is several orders of magnitude slower. Set dirty bit after write. • Write memory data back to disk if the page is candidate for replacement and its dirty bit is set. • When data are written back, the system can make effective use of a high bandwidth transfer because the entire block is transferred. The latency is amortized over many bytes in the block. • While old page are written back to the disk, processor can switch to other tasks until the new page have been fetched into the memory.

  39. Virtual Memory Summary • Virtual Memory is Another Level of the Memory Hierarchy • Creates the Illusion of a Very Large Memory • Page Tables, Possibly Multi-Level, Map a Virtual Address to a Physical Address • TLBs are Important for Fast Translation

More Related