Virtual Memory

1. Virtual Memory CSCI 444/544 Operating Systems Fall 2008

2. Agenda • What is virtual memory • rationales • intuition • Working mechanism of virtual memory • demand paging • page fault • Virtual memory policies • page selection • page replacement

3. What is Virtual Memory • OS goal: support processes when not enough physical memory • Single process with very large address space • Multiple processes with combined address spaces • Virtual Memory: OS provides illusion of more physical memory • User code should be independent of amount of physical memory • Only part of the program needs to be in memory for execution

4. Virtual Memory • Logical address space can therefore be much larger than physical address space • Rationales behind virtual memory • - locality of reference • - memory hierarchy (backing store, I.e., disk) • Paging makes virtual memory possible at fine-grain • - demand paging: make a physical instance of a page in memory only • when needed

5. Locality of Reference • Leverage locality of reference within processes • Spatial: reference memory addresses near previously referenced addresses • Temporal: reference memory addresses that have referenced in the past • Processes spend majority of time in small portion of code • Estimate: 90% of time in 10% of code • Implication: • Process only uses small amount of address space at any moment • Only small amount of address space must be resident in physical memory

6. Backing Store • Leverage memory hierarchy of machine architecture • Each layer acts as “backing store” for layer above • with virtual memory, the whole address space of each process has a copy in the backing store (I.e., disk) • consider the whole program actually resides on the disk, only part of it is cached in memory • with each page table entry, a valid-invalid bit is associated • 1 => in memory, 0 => not-in-memory or invalid logical page

7. Overview of Virtual Memory

8. Virtual Memory Intuition • Idea: OS keeps unreferenced pages on disk • Slower, cheaper backing store than memory • Process can run when not all pages are loaded into main memory • OS and hardware cooperate to provide illusion of large disk as fast as main memory • Same behavior as if all of address space in main memory • Hopefully have similar performance • Requirements: • OS must have mechanism to identify location of each page in address space in memory or on disk • OS must have policy for determining which pages live in memory and which on disk

9. Demand Paging • Bring a page into memory only when it is needed • Less I/O needed • Less memory needed • Faster response • More user programs running • Page is needed => reference to it • Invalid reference => abort • Not-in-memory => bring into memory • valid-invalid bit = 0 => page fault • Or some systems introduce extra-bit: present • present is clear => page fault

10. Virtual Memory Translation • Hardware and OS cooperate to translate addresses • First, hardware checks TLB for virtual address • if TLB hit, address translation is done; page in physical memory • If TLB miss... • Hardware or OS walk page tables • If valid-invalid (present) bit is 1, then page in physical memory • If page fault (i.e., valid-invalid bit is 0 or present bit is cleared) • Trap into OS • OS selects a free frame or a victim page in memory to replace • Write victim page out to disk if modified (add dirty bit to PTE) • OS reads referenced page from disk into memory • Page table is updated, present bit is set • Process continues execution

11. Page Fault • A reference to a page with valid bit set to 0 will trap to OS => page fault • OS looks at PCB to decide • - invalid reference => abort • - just no in memory • . Get free frame • . Swap into frame • . Reset tables • What if there is no free frame? • - evict a victim page in memory

12. Page Fault Overhead • page fault exception handling • [swap page out] • swap page in • restart user program

13. Virtual Memory Benefits • Virtual memory allows other benefits during process creation: • - Copy-on-Write • - Memory-Mapped Files

14. Copy-on-Write • Copy-on-Write (COW) allows both parent and child processes to initially sharethe same pages in memoryIf either process modifies a shared page, only then is the page copied • COW allows more efficient process creation as only modified pages are copied • Free pages are allocated from a pool of zeroed-out pages

15. Memory-Mapped Files • Memory-mapped file I/O allows file I/O to be treated as routine memory access by mapping a disk block to a page in memory • A file is initially read using demand paging. • - A certain portion of the file is read from the file system into a physical page. • - Subsequent reads/writes to/from the file are like ordinary memory accesses. • Simplifies file access by treating file I/O through memory rather than read() and write() system calls • Also allows several processes to map the same file allowing the pages in memory to be shared

16. Virtual Memory Policies • OS has two decisions on a page fault • Page selection • When should a page (or pages) on disk be brought into memory? • Two cases • When process starts, code pages begin on disk • As process runs, code and data pages may be moved to memory • Page replacement • Which resident page (or pages) in memory should be thrown out to disk? • Goal: Minimize number of page faults • Page faults require milliseconds to handle (reading from disk) • Implication: Plenty of time for OS to make good decision

17. Page Selection • When should a page be brought from disk into memory? • Demand paging: Load page only when a reference is made to location on that page (page fault occurs) • Intuition: Wait until page must absolutely be in memory • When process starts: No pages are loaded in memory • a flurry of page faults • after a time, locality works and reduces the page faults to a very low level • Pros: less work for user • Cons: pay cost of page fault for every newly accessed page • Pre-paging (anticipatory, pre-fetching): OS loads page into memory before page is referenced

18. Pre-Paging • OS predicts future accesses and brings pages into memory ahead of time • Works well for some access patterns (e.g., sequential) • Pros: May avoid page faults • Cons: ineffective if most of the extra pages that are brought in are not referenced. • Combine demand or pre-paging with user-supplied hints about page references • User specifies: may need page in future, don’t need this page anymore, or sequential access pattern, ...

19. What happens if no free frame available • Page replacement – find some page in memory, but not really in use, swap it out • Algorithm • FIFO • Optimal • Least Recently Used (LRU) • performance – want an algorithm which will result in minimum number of page faults

20. Page Replacement • Which page in main memory should selected as victim? • Write out victim page to disk if modified (dirty bit set) • If victim page is not modified (clean), just discard • A dirty bit for each page • Indicating if a page has been changed since last time loaded from the backing store • Indicating whether swap-out is necessary for the victim page

21. Page Replacement Algorithms • Page replacement algorithm: the algorithm that picks the victim page • Metrics: • Low page-fault rate • Implementation cost/feasibility • For the page-fault rate, we evaluate an algorithm • by running it on a particular string of memory references (reference string) • and computing the number of page faults on that string

22. First-In-First-Out (FIFO) • FIFO: Replace page that has been in memory the longest • Intuition: First referenced long time ago, done with it now • Advantages: • Fair: All pages receive equal residency • Easy to implement (circular buffer) • Disadvantage: Some pages may always be needed

23. A Case of FIFO • Reference string: 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5 • 3 frames (3 pages can be in memory at a time per process) • 4 frames • FIFO Replacement – Belady’s Anomaly • more frames => more page faults 1 1 4 5 2 2 1 3 9 page faults 3 3 2 4 1 1 5 4 2 2 1 10 page faults 5 3 3 2 4 4 3

24. Optimal Algorithm • Replace page that will not used for longest period of time in the future • Advantages: Guaranteed to minimize number of page faults • Disadvantages: Requires that OS predict the future • Not practical, but good for comparison • Used for measuring how well your algorithm performs

25. Least Recently Used (LRU) Algorithm • Replace page not used for longest time in past • Intuition: Use past to predict the future • past experience is a decent predictor of future behavior • Advantages: • With locality, LRU approximates OPT • Disadvantages: • Harder to implement, must track which pages have been accessed • Does not handle all workloads well

26. A Case of LRU • Reference string: 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5 • Counter implementation • Every page entry has a counter; every time page is referenced through this entry, copy the clock into the counter • When a page needs to be replaced, look at the counters to determine which one to replace 1 5 2 3 5 4 4 3

27. Stack Implementation of LRU • Stack implementation – keep a stack of page numbers in a double link form: • Page referenced: move it to the top • Always replace at the bottom of the stack • No search for replacement

28. Implementing LRU • Software Perfect LRU (stack-based) • OS maintains ordered list of physical pages by reference time • When page is referenced: Move page to front of list • When need victim: Pick page at back of list • Trade-off: Slow on memory reference, fast on replacement • Hardware Perfect LRU (clock-based) • Associate register with each page • When page is referenced: Store system clock in register • When need victim: Scan through registers to find oldest clock • Trade-off: Fast on memory reference, slow on replacement (especially as size of memory grows) • In practice, do not implement Perfect LRU • LRU is an approximation anyway, so approximate more • Goal: Find an old page, but not necessarily the very oldest

29. LRU Approximation Algorithms • Reference bit • With each page associate a bit, initially = 0 • When page is referenced bit set to 1 • Replace the one which is 0 (if one exists). We do not know the order, however. • Second chance • Need reference bit • Circle replacement • Logically arrange all physical frames in a big circle (clock) • If page to be replaced (in clock order) has reference bit = 1 then: • set reference bit 0 • leave page in memory • replace next page (in clock order), subject to same rules

30. Second-chance Algorithm

31. Page Replacement Comparison • Add more physical memory, what happens to performance? • LRU, OPT: Add more memory, guaranteed to have fewer (or same number of) page faults • Smaller memory sizes are guaranteed to contain a subset of larger memory sizes • FIFO: Add more memory, usually have fewer page faults • Belady’s anomaly: May actually have more page faults!