C SINGH, JUNE 7-8, 2010

Advanced Computers Architecture Virtual Memory By Rohit Khokher Department of Computer Science, Sharda University, Greater Noida, India C SINGH, JUNE 7-8, 2010 Advanced Computers Architecture, UNIT 2 IWW 2010, ISATANBUL, TURKEY

Topic s of my Talk • 1.Principles of Virtual Memory • 2. Implementations of Virtual Memory • Paging • Segmentation • Paging With Segmentation • Paging of System Tables • Translation Look-aside Buffers • 3. Memory Allocation in Paged Systems • Global Page Replacement Algorithms • Local Page Replacement Algorithms • Load Control and Thrashing • Evaluation of Paging C SINGH, JUNE 7-8, 2010 Advanced Computers Architecture, UNIT 2 IWW 2010, ISATANBUL, TURKEY

Hardware and Control Structures • Memory references are dynamically translated into physical addresses at run time • A process may be swapped in and out of main memory such that it occupies different regions • A process may be broken up into pieces that do not need to located contiguously in main memory • All pieces of a process do not need to be loaded in main memory during execution C SINGH, JUNE 7-8, 2010 Advanced Computers Architecture, UNIT 2 IWW 2010, ISATANBUL, TURKEY

Execution of a Program • Operating system brings into main memory a few pieces of the program • Resident set - portion of process that is in main memory • An interrupt is generated when an address is needed that is not in main memory • Operating system places the process in a blocking state C SINGH, JUNE 7-8, 2010 Advanced Computers Architecture, UNIT 2 IWW 2010, ISATANBUL, TURKEY

Piece of process that contains the logical address is brought into main memory • Operating system issues a disk I/O Read request • Another process is dispatched to run while the disk I/O takes place • An interrupt is issued when disk I/O complete which causes the operating system to place the affected process in the Ready state C SINGH, JUNE 7-8, 2010 Advanced Computers Architecture, UNIT 2 IWW 2010, ISATANBUL, TURKEY

Advantages of Breaking up a Process • More processes may be maintained in main memory • Only load in some of the pieces of each process • With so many processes in main memory, it is very likely a process will be in the Ready state at any particular time • A process may be larger than all of main memory C SINGH, JUNE 7-8, 2010 Advanced Computers Architecture, UNIT 2 IWW 2010, ISATANBUL, TURKEY

Types of Memory • Real memory • Main memory • Virtual memory • Memory on disk • Allows for effective multiprogramming and relieves the user of tight constraints of main memory C SINGH, JUNE 7-8, 2010 Advanced Computers Architecture, UNIT 2 IWW 2010, ISATANBUL, TURKEY

Virtual Memory Virtual memory is a technique that allows processes that may not be entirely in the memory to execute by means of automatic storage allocation upon request. The term virtual memory refers to the abstraction of separating LOGICAL memory--memory as seen by the process--from PHYSICAL memory--memory as seen by the processor. Because of this separation, the programmer needs to be aware of only the logical memory space while the operating system maintains two or more levels of physical memory space. The virtual memory abstraction is implemented by using secondary storage to augment the processor's main memory. Data is transferred from secondary to main storage as and when necessary and the data replaced is written back to the secondary storage according to a predetermined replacement algorithm. If the data swapped is designated a fixed size, this swapping is called paging; if variable sizes are permitted and the data is split along logical lines such as subroutines or matrices, it is called segmentation. Some operating systems combine segmentation and paging. C SINGH, JUNE 7-8, 2010 Advanced Computers Architecture, UNIT 2 IWW 2010, ISATANBUL, TURKEY

The diagram illustrates that a program generated address ( 1 ) or "logical address" consisting of a logical page number plus the location within that page (x) must be interpreted or "mapped" onto an actual (physical) main memory address by the operating system using an address translation function or mapper ( 2 ). If the page is present in the main memory, the mapper substitutes the physical page frame number for the logical number ( 3 ). If the mapper detects that the page requested is not present in main memory, a fault occurs and the page must be read into a frame in main memory from secondary storage C SINGH, JUNE 7-8, 2010 Advanced Computers Architecture, UNIT 2 IWW 2010, ISATANBUL, TURKEY

Assignment What do you think are the main considerations in implementing this virtual memory system? C SINGH, JUNE 7-8, 2010 Advanced Computers Architecture, UNIT 2 IWW 2010, ISATANBUL, TURKEY

A System with Physical Memory Only CPU • Examples: • most Cray machines, early PCs, nearly all embedded systems, etc. Memory 0: Physical Addresses 1: N-1: Addresses generated by the CPU point directly to bytes in physical memory C SINGH, JUNE 7-8, 2010 Advanced Computers Architecture, UNIT 2 IWW 2010, ISATANBUL, TURKEY

A System with Virtual Memory 0: 1: CPU N-1: • Examples: • workstations, servers, modern PCs, etc. Memory Page Table Virtual Addresses Physical Addresses 0: 1: P-1: Disk Address Translation: Hardware converts virtual addresses to physical addresses via an OS-managed lookup table (page table) Advanced Computers Architecture, UNIT 2 C SINGH, JUNE 7-8, 2010 IWW 2010, ISATANBUL, TURKEY

Page Faults (Similar to “Cache Misses”) • What if an object is on disk rather than in memory? • Page table entry indicates virtual address not in memory • OS exception handler invoked to move data from disk into memory • current process suspends, others can resume • OS has full control over placement, etc. Before fault After fault Memory Memory Page Table Page Table Virtual Addresses Physical Addresses Virtual Addresses Physical Addresses CPU CPU Disk Disk Advanced Computers Architecture, UNIT 2 C SINGH, JUNE 7-8, 2010 IWW 2010, ISATANBUL, TURKEY

Mapper The mapper is the part of the operating system that translates the logical page number generated by the program into the physical page frame number where the main memory holds the page. This translation is accomplished by using a directly indexed table called the page table which identifies the location of all the program's pages in the main store. If the page table reveals that the page is, in fact, not resident in the main memory, the mapper issues a page fault to the operating system so that execution is suspended on the process until the desired page can be read in from the secondary store and placed in main memory. C SINGH, JUNE 7-8, 2010 Advanced Computers Architecture, UNIT 2 IWW 2010, ISATANBUL, TURKEY

Terms & Notions • Virtual memory (VM) is • Not a physical device but an abstract concept • Comprised of the virtual address spaces (of all processes) • Virtual address space (VAS) (of one process) • Set of visible virtual addresses • (Some systems may use a single VAS for all processes) • Resident set • Pieces of a process currently in physical memory • Working set • Set of pieces a process is currently working on Advanced Computers Architecture, UNIT 2

The Principle of Locality repeated references: initialization data initialization code finalization code code 1 code 2 data single jumps: working set: early phase of process lifetime final phase of process lifetime main phase of process lifetime • Principle of Locality: • Memory references within a process tend to cluster • Working set should be part of the resident set to operate efficiently (else: frequent memory access faults) honor the principle of locality to achieve this Advanced Computers Architecture, UNIT 2

VM: FeaturesSwapping „piece“ modified? find rarely used „piece" adjust mapping table no discard „piece“ yes save HDD location of „piece“ write „piece“ out to disk • Danger: Thrashing: • „Piece“ just swapped out is immediately requested again • System swaps in/out all the time, no real work is done • Thus: „piece“for swap out has to be chosen carefully • Keep track of „piece“ usage („age of piece“) • Hopefully „piece“ used frequently lately will be used again in near future (principle of locality!) lack of memory no need to swap out complete process!!! Advanced Computers Architecture, UNIT 2

VM: FeaturesProtection • Each process has its own virtual address space • Processes invisible to each other • Process cannot access another processes memory • MMU checks protection bits on memory access (during address mapping) • „Pieces“ can be protected from being written to or being executed or even being read • System can distinguish different protection levels (user / kernel mode) • Write protection can be used to implement copy on write ( Sharing) Advanced Computers Architecture, UNIT 2

VM: FeaturesSharing Piece0 Piece0 Piece 0 Piece 1 Piece1 Piece1 Piece 2 Piece2 Piece2 Physical memory Virtual memory Process 1 Virtual memory Process 2 • Pieces of different processes mapped to one single „piece“ of physical memory • Allows sharing of code (saves memory), e.g. libraries • Copy on write: „piece“ may be used by several processes until one writes to it (then that process gets its own copy) • Simplifies interprocess-communication (IPC) shared memory Advanced Computers Architecture, UNIT 2

VM: Advantages (1) • VM supports • Swapping • Rarely used „pieces“ can be discarded or swapped out • „Piece“ can be swapped back in to any free piece of physical memory large enough, mapping unit translates addresses • Protection • Sharing • Common data or code may be shared to save memory • Process need not be in memory as a whole • No need for complicated overlay techniques (OS does job) • Process may even be larger than all of physical memory • Data / code can be read from disk as needed Advanced Computers Architecture, UNIT 2

VM: Advantages (2) • Code can be placed anywhere in physical memory without relocation (adresses are mapped!) • Increased cpu utilization • more processes can be held in memory (in part) more processes in ready state (consider: 80% HDD I/O wait time not uncommon) Advanced Computers Architecture, UNIT 2

VM: Disadvantages • Memory requirements (mapping tables) • Longer memory access times (mapping table lookup) • Can beimproved usingTLB Advanced Computers Architecture, UNIT 2

VM: Implementation • VM may be implemented using • Paging • Segmentation • Combination of both Advanced Computers Architecture, UNIT 2

Address Translation • Address translation must be fast (it happens on every memory access). • We need a fully associative placement policy. • miss penalty is huge! • We can’t afford to go looking at every virtual page to find the right one • we don’t use the tag bits approach C SINGH, JUNE 7-8, 2010 Advanced Computers Architecture, UNIT 2 IWW 2010, ISATANBUL, TURKEY

VM Address Translation V = {0, 1, . . . , N–1} virtual address space P = {0, 1, . . . , M–1} physical address space MAP: V  P U {} address mapping function N > M MAP(a) = a' if data at virtual address a is present at physical address a' in P =  if data at virtual address a is not present in P page fault fault handler Processor  Hardware Addr Trans Mechanism Secondary memory Main Memory a a' OS performs this transfer (only if miss) virtual address part of the on-chip memory mgmt unit (MMU) physical address Advanced Computers Architecture, UNIT 2 C SINGH, JUNE 7-8, 2010 IWW 2010, ISATANBUL, TURKEY

VM Address Translation • Parameters • P = 2p = page size (bytes). • N = 2n = Virtual address limit • M = 2m = Physical address limit n–1 p p–1 0 virtual address virtual page number page offset address translation m–1 p p–1 0 physical address physical page number page offset Notice that the page offset bits don't change as a result of translation Advanced Computers Architecture, UNIT 2 C SINGH, JUNE 7-8, 2010 IWW 2010, ISATANBUL, TURKEY

Page Tables Memory resident page table (physical page or disk address) Virtual Page Number Physical Memory Valid 1 1 0 1 1 1 0 1 Disk Storage (swap file or regular file system file) 0 1 Advanced Computers Architecture, UNIT 2 C SINGH, JUNE 7-8, 2010 IWW 2010, ISATANBUL, TURKEY

Address Translation via Page Table virtual address page table base register n–1 p p–1 0 VPN acts as table index virtual page number (VPN) page offset physical page number (PPN) access valid if valid=0 then page not in memory m–1 p p–1 0 physical page number (PPN) page offset physical address Advanced Computers Architecture, UNIT 2 C SINGH, JUNE 7-8, 2010 IWW 2010, ISATANBUL, TURKEY

Page Table Operation • Translation • Separate (set of) page table(s) per process • VPN forms index into page table (points to a page table entry) • Computing Physical Address • Page Table Entry (PTE) provides information about page • if (valid bit = 1) then the page is in memory. • Use physical page number (PPN) to construct address • if (valid bit = 0) then the page is on disk • Page fault • Must load page from disk into main memory before continuing • Checking Protection • Access rights field indicate allowable access • e.g., read-only, read-write, execute-only • typically support multiple protection modes (e.g., kernel vs. user) • Protection violation fault if user doesn’t have necessary permission C SINGH, JUNE 7-8, 2010 Advanced Computers Architecture, UNIT 2 IWW 2010, ISATANBUL, TURKEY

Integrating VM and Cache miss VA PA Trans- lation Cache Main Memory CPU hit data • Most Caches “Physically Addressed” • Accessed by physical addresses • Allows multiple processes to have blocks in cache at same time • Allows multiple processes to share pages • Cache doesn’t need to be concerned with protection issues • Access rights checked as part of address translation • Perform Address Translation Before Cache Lookup • But this could involve a memory access itself (of the PTE) • Of course, page table entries can also become cached C SINGH, JUNE 7-8, 2010 Advanced Computers Architecture, UNIT 2 IWW 2010, ISATANBUL, TURKEY

Speeding up Translation with a TLB hit miss VA PA TLB Lookup Cache Main Memory CPU miss hit Trans- lation data • “Translation Lookaside Buffer” (TLB) • Small hardware cache in MMU • Maps virtual page numbers to physical page numbers • Contains complete page table entries for small number of pages Advanced Computers Architecture, UNIT 2 C SINGH, JUNE 7-8, 2010 IWW 2010, ISATANBUL, TURKEY

Address Translation with a TLB n–1 p p–1 0 virtual address virtual page number page offset valid tag physical page number TLB . . . = TLB hit physical address tag byte offset index valid tag data Cache = data cache hit Advanced Computers Architecture, UNIT 2 C SINGH, JUNE 7-8, 2010 IWW 2010, ISATANBUL, TURKEY

Simple Memory System Example 8 5 6 7 11 9 10 12 13 4 1 2 3 10 0 0 1 2 3 4 11 6 7 8 9 5 VPO VPN • Addressing • 14-bit virtual addresses • 12-bit physical address • Page size = 64 bytes PPN PPO (Virtual Page Offset) (Virtual Page Number) (Physical Page Number) (Physical Page Offset) C SINGH, JUNE 7-8, 2010 Advanced Computers Architecture, UNIT 2 IWW 2010, ISATANBUL, TURKEY

Simple Memory System Page Table • Only show first 16 entries C SINGH, JUNE 7-8, 2010 Advanced Computers Architecture, UNIT 2 IWW 2010, ISATANBUL, TURKEY

C SINGH, JUNE 7-8, 2010 Advanced Computers Architecture, UNIT 2 IWW 2010, ISATANBUL, TURKEY

C SINGH, JUNE 7-8, 2010