Virtual Memory

1. Virtual Memory

2. Virtual Memory Manager • Provides abstraction of physical memory • Creates virtual address space in secondary memory and then “automatically” determines which part of the address space to load into primary memory at any given time • Allows application programmers to think that they have a very large address space in which to write programs

3. Virtual Memory Organization Primary Memory Secondary Memory Memory Image for pi

4. Locality • Programs do not access their address space uniformly • they access the same location over and over • Spatial locality: processes tend to access location near to location they just accessed • because of sequential program execution • because data for a function is grouped together • Temporal locality: processes tend to access data over and over again • because of program loops • because data is processed over and over again

5. <1% 15% 35% 20% <1% 30% Execution time Spatial Reference Locality • Address space is logically partitioned • Text, data, stack • Initialization, main, error handle • Different parts have different reference patterns: Address Space for pi Initialization code (used once) Code for 1 Code for 2 Code for 3 Code for error 1 Code for error 2 Code for error 3 Data & stack

6. Virtual Memory • Every process has code and data locality • Dynamically load/unload currently-used address space fragments as the process executes • Uses dynamic address relocation/binding • Generalization of base-limit registers • Physical address corresponding to a compile-time address is not bound until run time

7. Virtual Memory – cont • Since binding changes with time, use a dynamic virtual address map, Yt Virtual Address Space Yt

8. Physical Address Space 0 n-1 • Each address space is fragmented Primary Memory • Fragments of the virtual address space are dynamically loaded into primary memory at any given time Virtual Memory – cont Secondary Memory Virtual Address Space for pi Virtual Address Space for pj Virtual Address Space for pk • Complete virtual address space is stored in secondary memory

9. Address Translation • Virtual memory systems distinguish among symbolic names, virtual address, and physical address spaces • Need to map symbolic names to virtual addresses, and then to physical addresses • Compiler/assembler and link editor handle mapping from symbolic names in name space to virtual address • When program is executed, the virtual addresses are mapped to physical addresses

10. Executable Image Physical Address Space Yt: Virtual Address Space  Physical Address Space Names, Virtual Addresses & Physical Addresses Dynamically Source Program Absolute Module Name Space Pi’s Virtual Address Space

11. Address Formation • Translation system creates an address space, but its address are virtual instead of physical • A virtual address, x: • Is mapped to physical address y = Yt(x) if x is loaded at physical address y • Is mapped to W (the null address) if x is not loaded • The map, Yt, changes as the process executes -- it is “time varying” • Yt: Virtual Address  Physical Address  {W}

12. Translation Process • If Yt(k) = W at time t and the process references location k, then • The virtual manager will stop the process • The referenced location is loaded at some location (say m) • The manager changes Yt(k) = m • The manager lets the process continue execution • Note that the referenced element was determined missing after an instruction started execution – CPU needs to be able to “back out” of an instruction and reexecute instruction after translation mapping

13. Size of Blocks of Memory • Virtual memory system transfers “blocks” of the address space to/from primary memory • Fixed size blocks: System-defined pages are moved back and forth between primary and secondary memory • Variable size blocks: Programmer-defined segments – corresponding to logical fragments – are the unit of movement • Paging is the commercially dominant form of virtual memory today

14. Paging • A page is a fixed size, 2h, block of virtual addresses • A page frame is a fixed size, 2h, block of physical memory (the same size as a page) • When a virtual address, x, in page i is referenced by the CPU • If page i is loaded at page frame j, the virtual address is relocated to page frame j • If page is not loaded, the OS interrupts the process and loads the page into a page frame

15. Practicality of paging • Paging only works because of locality • at any one point in time programs don’t need most of their pages • Page fault rates must be very, very low for paging to be practical • like one page fault per 100,000 or more memory references

16. Addresses • Suppose there are G= 2g2h=2g+h virtual addresses and H=2j+h physical addresses assigned to a process • Each page/page frame is 2h addresses • There are 2g pages in the virtual address space • 2j page frames are allocated to the process • Rather than map individual addresses • Yt maps the 2g pages to the 2j page frames • That is, page_framej = Yt(pagei) • Address k in pagei corresponds to address k in page_framej

17. Page-Based Address Translation • Let N = {d0, d1, … dn-1} be the pages • Let M = {b0, b1, …, bm-1} be page frames • Virtual address, i, satisfies 0i<G= 2g+h • Physical address, k = U2h+V (0V<G= 2h ) • U is page frame number • V is the line number within the page • Yt:[0:G-1]  <U, V>  {W} • Since every page is size c=2h • page number = U = i/c • line number = V = i mod c

18. Address Translation (cont) g bits h bits Virtual Address Page # Line # “page table” Missing Page Yt j bits h bits Physical Address Frame # Line # CPU Memory MAR

19. Paging Algorithms • Two basic types of paging algorithms • Static allocation • Dynamic allocation • Three basic policies in defining any paging algorithm • Fetch policy – when a page should be loaded • Replacement policy –which page is unloaded • Placement policy – where page should be loaded

20. Fetch Policy • Determines when a page should be brought into primary memory • Usually don’t have prior knowledge about what pages will be needed • Majority of paging mechanisms use a demand fetch policy • Page is loaded only when process references it

21. Demand Paging Algorithm • Page fault occurs • Process with missing page is interrupted • Memory manager locates the missing page • Page frame is unloaded (replacement policy) • Page is loaded in the vacated page frame • Page table is updated • Process is restarted

22. Page references • Processes continually reference memory • and so generate a stream of page references • The page reference stream tells us everything about how a process uses memory • For a given size, we only need to consider the page number • If we have a reference to a page, then immediately following references to the page will never generate a page fault 0100, 0432, 0101, 0612, 0102, 0103, 0104, 0101, 0611, 0102, 0103 0104, 0101, 0610, 0103, 0104, 0101, 0609, 0102, 0105 • Suppose the page size is 100 bytes, what is the page reference stream? • We use page reference streams to evaluate paging algorithms

23. Modeling Page Behavior • Let w = r1, r2, r3, …, ri, … be a page reference stream • ri is the ith page # referenced by the process • The subscript is the virtual time for the process • Given a page frame allocation of m, the memory state at time t, St(m), is set of pages loaded • St(m) = St-1(m)  Xt - Yt • Xt is the set of fetched pages at time t • Yt is the set of replaced pages at time t

24. More on Demand Paging • If rt was loaded at time t-1, St(m) = St-1(m) • If rt was not loaded at time t-1 and there were empty page frames • St(m) = St-1(m)  {rt} • If rt was not loaded at time t-1 and there were no empty page frames • St(m) = St-1(m)  {rt} - {y} where y is the page frame unloaded

25. Replacement Policy • When there is no empty page frame in memory, we need to find one to replace • Write it out to the swap area if it has been changed since it was read in from the swap area • Dirty bit or modified bit • pages that have been changed are referred to as “dirty” • these pages must be written out to disk because the disk version is out of date • this is called “cleaning” the page • Which page to remove from memory to make room for a new page • We need a page replacement algorithm

26. Page replacement algorithms • The goal of a page replacement algorithm is to produce the fewest page faults • We can compare two algorithms • on a range of page reference streams • Or we can compare an algorithm to the best possible algorithm • We will start by considering static page replacement algorithms

27. Static Paging Algorithms • A fixed number of page frames is allocated to each process when it is created • Paging policy defines how these page frames will be loaded and unloaded • Placement policy is fixed • The page frame holding the new page is always the one vacated by the page selected for replacement

28. Static Allocation, Demand Paging • Number of page frames is static over the life of the process • Fetch policy is demand • Since St(m) = St-1(m)  {rt} - {y}, the replacement policy must choose y -- which uniquely identifies the paging policy

29. Random page replacement • Algorithm: replace a page randomly • Theory: we cannot predict the future at all • Implementation: easy • Performance: poor • but it is easy to implement • but the best case, worse case and average case are all the same

30. 13 page faults • No knowledge of v doesn’t perform well Random Replacement • Replaced page, y, is chosen from the m loaded page frames with probability 1/m Let page reference stream, v = 2031203120316457 Frame 2 0 3 1 2 0 3 1 2 0 3 1 6 4 5 7 0 1 2 2 2 0 2 0 3 2 1 3 2 1 3 2 1 0 3 1 0 3 1 0 3 1 2 0 1 2 0 3 2 0 3 2 0 6 2 4 6 2 4 5 2 7 5 2

31. Belady’s Optimal algorithm • The one that produces the fewest possible page faults on all page reference sequences • Algorithm: replace the page that will not be used for the longest time in the future • Problem: it requires knowledge of the future • Not realizable in practice • but it is used to measure the effectiveness of realizable algorithms

32. Belady’s Optimal Algorithm • Replace page with maximal forward distance: yt = max xeS t-1(m)FWDt(x) Let page reference stream, v = 2031203120316457 Frame 2 0 3 1 2 0 3 1 2 0 3 1 6 4 5 7 0 2 2 2 1 0 0 2 3

33. Belady’s Optimal Algorithm • Replace page with maximal forward distance: yt = max xeS t-1(m)FWDt(x) Let page reference stream, v = 2031203120316457 Frame 2 0 3 1 2 0 3 1 2 0 3 1 6 4 5 7 0 2 2 2 1 0 0 2 3 FWD4(2) = 1 FWD4(0) = 2 FWD4(3) = 3

34. Belady’s Optimal Algorithm • Replace page with maximal forward distance: yt = max xeS t-1(m)FWDt(x) Let page reference stream, v = 2031203120316457 Frame 2 0 3 1 2 0 3 1 2 0 3 1 6 4 5 7 0 2 2 2 2 1 0 0 0 2 31 FWD4(2) = 1 FWD4(0) = 2 FWD4(3) = 3

35. Belady’s Optimal Algorithm • Replace page with maximal forward distance: yt = max xeS t-1(m)FWDt(x) Let page reference stream, v = 2031203120316457 Frame 2 0 3 1 2 0 3 1 2 0 3 1 6 4 5 7 0 2 2 2 2 2 2 1 0 0 0 0 0 2 31 1 1

36. Belady’s Optimal Algorithm • Replace page with maximal forward distance: yt = max xeS t-1(m)FWDt(x) Let page reference stream, v = 2031203120316457 Frame 2 0 3 1 2 0 3 1 2 0 3 1 6 4 5 7 0 2 2 2 2 2 2 2 1 0 0 0 0 0 3 2 31 1 1 1 FWD7(2) = 2 FWD7(0) = 3 FWD7(1) = 1

37. Belady’s Optimal Algorithm • Replace page with maximal forward distance: yt = max xeS t-1(m)FWDt(x) Let page reference stream, v = 2031203120316457 Frame 2 0 3 1 2 0 3 1 2 0 3 1 6 4 5 7 0 2 2 2 2 2 2 2 2 2 0 1 0 0 0 0 0 3 3 3 3 2 31 1 1 1 1 1 1 FWD10(2) =  FWD10(3) = 2 FWD10(1) = 3

38. Belady’s Optimal Algorithm • Replace page with maximal forward distance: yt = max xeS t-1(m)FWDt(x) Let page reference stream, v = 2031203120316457 Frame 2 0 3 1 2 0 3 1 2 0 3 1 6 4 5 7 0 2 2 2 2 2 2 2 2 2 0 0 0 1 0 0 0 0 0 3 3 3 3 3 3 2 31 1 1 1 1 1 1 1 1 FWD13(0) =  FWD13(3) =  FWD13(1) = 

39. Belady’s Optimal Algorithm • Replace page with maximal forward distance: yt = max xeS t-1(m)FWDt(x) Let page reference stream, v = 2031203120316457 Frame 2 0 3 1 2 0 3 1 2 0 3 1 6 4 5 7 0 2 2 2 2 2 2 2 2 2 0 0 0 0 4 4 4 1 0 0 0 0 0 3 3 3 3 3 3 6 6 6 7 2 31 1 1 1 1 1 1 1 1 1 1 5 5 10 page faults • Perfect knowledge of v perfect performance • Impossible to implement

40. Theories of program behavior • All replacement algorithms try to predict the future and act like Belady’s optimal algorithm • All replacement algorithms have a theory of how program behave • they use it to predict the future, that is, when pages will be referenced • then the replace the page that they think won’t be referenced for the longest time.

41. LRU page replacement • Least-recently used (LRU) • Algorithm: remove the page that hasn’t been referenced for the longest time • Theory: the future will be like the past, page accesses tend to be clustered in time • Implementation: hard, requires hardware assistance (and then still not easy) • Performance: very good, within 30%-40% of optimal

42. LRU model of the future

43. Least Recently Used (LRU) • Replace page with maximal forward distance: yt = max xeS t-1(m)BKWDt(x) Let page reference stream, v = 2031203120316457 Frame 2 0 3 1 2 0 3 1 2 0 3 1 6 4 5 7 0 2 2 2 1 0 0 2 3 BKWD4(2) = 3 BKWD4(0) = 2 BKWD4(3) = 1

44. Least Recently Used (LRU) • Replace page with maximal forward distance: yt = max xeS t-1(m)BKWDt(x) Let page reference stream, v = 2031203120316457 Frame 2 0 3 1 2 0 3 1 2 0 3 1 6 4 5 7 0 2 2 2 1 1 0 0 0 2 3 3 BKWD4(2) = 3 BKWD4(0) = 2 BKWD4(3) = 1

45. Least Recently Used (LRU) • Replace page with maximal forward distance: yt = max xeS t-1(m)BKWDt(x) Let page reference stream, v = 2031203120316457 Frame 2 0 3 1 2 0 3 1 2 0 3 1 6 4 5 7 0 2 2 2 1 1 1 0 0 0 2 2 3 3 3 BKWD5(1) = 1 BKWD5(0) = 3 BKWD5(3) = 2

46. Least Recently Used (LRU) • Replace page with maximal forward distance: yt = max xeS t-1(m)BKWDt(x) Let page reference stream, v = 2031203120316457 Frame 2 0 3 1 2 0 3 1 2 0 3 1 6 4 5 7 0 2 2 2 1 1 1 1 0 0 0 2 2 2 3 3 3 0 BKWD6(1) = 2 BKWD6(2) = 1 BKWD6(3) = 3

47. Least Recently Used (LRU) • Replace page with maximal forward distance: yt = max xeS t-1(m)BKWDt(x) Let page reference stream, v = 2031203120316457 Frame 2 0 3 1 2 0 3 1 2 0 3 1 6 4 5 7 0 2 2 2 1 1 1 3 3 3 0 0 0 6 6 6 7 1 0 0 0 2 2 2 1 1 1 3 3 3 4 4 4 2 3 3 3 0 0 0 2 2 2 1 1 1 5 5

48. Least Recently Used (LRU) • Replace page with maximal forward distance: yt = max xeS t-1(m)BKWDt(x) Let page reference stream, v = 2031203120316457 Frame 2 0 3 1 2 0 3 1 2 0 3 1 6 4 5 7 0 2 2 2 2 2 2 2 3 2 2 2 2 6 6 6 6 1 0 0 0 0 0 0 0 0 0 0 0 0 4 4 4 2 3 3 3 3 3 3 3 3 3 3 3 3 5 5 3 1 1 1 1 1 1 1 1 1 1 1 1 7 • Backward distance is a good predictor of forward distance -- locality

49. LFU page replacement • Least-frequently used (LFU) • Algorithm: remove the page that hasn’t been used often in the past • Theory: an actively used page should have a large reference count • Implementation: hard, also requires hardware assistance (and then still not easy) • Performance: not very good

50. Least Frequently Used (LFU) • Replace page with minimum use: yt = min xeS t-1(m)FREQ(x) Let page reference stream, v = 2031203120316457 Frame 2 0 3 1 2 0 3 1 2 0 3 1 6 4 5 7 0 2 2 2 1 0 0 2 3 FREQ4(2) = 1 FREQ4(0) = 1 FREQ4(3) = 1