More Multiprogramming Issues

More Multiprogramming Issues ECEN5043 Software Engineering of Multi-program Systems University of Colorado, Boulder

Quote-of-the-week award “Memory is an important resource that must be carefully managed.” A. S. Tanenbaum :-)) ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Overview -- Resource Management • Memory management • If certain techniques are not relevant now, they may be in the future as they were once before -- historical cycles • Input/Output • Understand impact of choices • Device independence; process independence • Shared resources • Spooling • File Systems • Multimedia (high demand, real time) file issues ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Growing pains • In a given environment, programs get bigger faster than memories • In other words, software expands to “max out” available hardware resources -- memory is one of those ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Memory Management Concerns • Which parts of a memory are in use • Allocate memory to processes when needed • De-allocate memory when processes do not need it • Manage swapping between hierarchies of memory as needed • onboard cache • separate cache (possibly multi-layer) • main memory • disk ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Simple is as simple does ... • There are simple schemes and complicated ones. Depending on the program’s environment, • the simpler ones are in use or obsolete • the more complicated ones are • in use • not thought of yet • or not possible to apply ... yet • What environments for which? • Why? ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Monoprogramming • Run one program at a time • Share memory between the program and the operating system (may be primitive) • First used on mainframes, then minis, then PCs, then palmtops and simple embedded systems ... • What conditions make it appropriate? User program Op. Sys. in RAM Op. Sys. in ROM User program Device drivers in ROM User program Op. Sys. in RAM ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Multiprogramming • Multiple processes ready to run simultaneously means • When one process is blocked waiting for I/O to finish, another one can use the CPU • Increased CPU utilization • Expectations of this capability are increasing • Network servers -- expected to run (serve) multiple processes • Client machines also have this ability ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Some ways to achieve multiprogramming • Easiest: divide memory into n partitions, not necessarily equal • Each process has a queue • Put a new process into input queue for smallest partition large enough to hold it (problem? remedy?) • One queue fits all • Scan jobs in an order-of-arrival queue; select first fit for available partition (problem? remedy?) • Scan all; select best fit (problem? remedy?) ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Modeling Multiprogramming • Probabilistic viewpoint • Process spends fraction p of its time waiting for I/O to complete • n processes in memory at once -- probability that all n processes are waiting for I/O is pn • Prob. that CPU is being used: CPU utilization = 1 - pn • Interactive systems, batch systems with disk I/O, (others?) can have 80% wait time • Assumptions don’t quite fit -- makes this an approximation, not an accurate calculation ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Analysis of Multiprogramming System Performance • Analysis examples are in the online handout in 4.1.4, p. 194 • You have a homework problem, too. • If you designed a system of programs and those programs will be the only ones running in a certain environment ... • What information do you need to know to be able to do an analysis of performance? • How can you get that information? ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Multiprogramming Introduces 2 Problems • Protection -- the other problem • A malicious program can construct a new instruction and jump to it • If programs use absolute addresses rather than relative addresses, difficult to stop it • Protection-code solution • Base/limit approach solved both the relocation and protection problems • base has address of partition start; limit has length • disadvantage? ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

When is “simple” not enough? • Simplest memory management to implement • fixed partitions • process/job loaded into partition when it gets to the head of the queue • stays in memory until done • What questions do you want to know the answers to re user-threads or user-spawned copies of processes? • Why do anything different? ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Whenever you open a can of worms, it always requires a bigger can to put them back. from Murphy’s Law and Other Reasons Why Things Go Wrong wrong

Pandora’s box syndrome ... • Swapping • Bring in a process in its entirety, run it, put back on disk if it has changed • Fixed partitions -- simplest • variable partitions -- much more complicated (see next slide) • Virtual memory • Bring in a portion of the process that includes what is needed • Once it is understood that only a portion needs to be in main memory, multiple portions can be present without being contiguous ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Swap Meet -- Variable Partitions In (g), A’s addresses must be relocated by software when it is swapped in or, more likely, by hardware during execution. ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Swapping with variable partitions • Number, location, and size of partitions vary dynamically • Improved memory utilization because not tied to fixed partitions that waste space or are too small • Complicates allocating and deallocating memory - why? • Complicates keeping track of available space - why? ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Managing a Growing Process • IF processes have fixed sizes, allocate what is needed. • IF data segment can grow • Process can grow into adjacent hole, if any • Process can be swapped to new location large enough • One or more other processes can be swapped out to create a large enough hole • Process can wait • Process can be killed ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

How much is enough? • If most processes will grow while running, design o.s. so that it • Allocates space in main memory a little more than is apparently needed • When swapping out, swap only memory actually in use OR • If supported, allow stack and heap to grow toward each other where the available memory in between is used by whichever needs it. ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Dynamic memory assignment • Managed by the operating system if there is one • You may be designing the technique if your company’s product is a very small computer (palm) or embedded system with no o.s. or a limited one. • Bitmap • Linked list -- first, next, best, worst, quick fit ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Linked Lists ... “Tables” • Linked list of allocated and free memory segments containing either processes or holes • First fit -- take first hole big enough, even if tooooo big • allocate the part that is needed; leaving new smaller hole • Next fit -- Keeps track of location when it finds a suitable hole -- searches from that point next time. • Best fit -- Searches all of list; takes the closest that is larger • Worst fit -- leaves the biggest hole possible • Quick fit -- separate lists of commonly requested sizes -- fast to find -- slow to merge remaining holes ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Overlays • Split the program into pieces called overlays; scheme for how this was done was designed by the programmer • When one overlay done, it called another (or other schemes) • Overlays kept on disk, swapped in and out of memory as needed • Eventually, operating systems took over the whole job with a technique called virtual memory • In the absence of an operating system, it is still a viable concept in specialized environments ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Virtual Memory • O.S. (or equivalent) keeps • parts of the program currently in use in main memory, not necessarily contiguously • parts of the program not currently in use (or blocked) may be on disk if the space is needed • may do this simultaneously for several programs • Paging, segmentation, paged-segmentation, etc. ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Simple Paging virtual address page # page offset page table pointer present frame n modified page table ... page table index page frame number physical address frame 2 frame 1 main memory ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Managing Page Tables • Page tables usually kept in memory -- they’re big • Execution speed is usually limited by the rate the CPU can get instructions and data out of memory • For each memory address reference, there is (theoretically) a page table memory reference plus the resulting memory reference • Locality of reference -- a small number of page table entries are heavily used; others rarely • Translation Lookaside Buffer is inside MMU with the frequently referenced page table entries’ information ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Software TLB Manager • RISC machines (Reduced Instruction Set Computer) • Small number of simple instructions • Programs are longer in number of lines but usually execute faster on simpler architectures • Burden is on the programmer (compiler or assembly language writer) to express the thought with a more restricted set of instructions • Tend to handle the TLB faults in the o.s. (software) instead of the MMU (hardware) ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Inverted Page Tables • When the virtual address space is too huge, the page table is too huge -- one entry for each page. • Instead, use a table with an entry for each physical frame • Keeps track of which process and virtual page is located in the page frame • Plus: save lots of space when the virtual address space is much larger than the physical address space • Minus: Virtual-to-physical translation becomes much harder • Use TLB for the inverted page table ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Page Replacement - a kind of prioritization • Other techniques for dealing with large virtual memories – pdf excerpt from book gives references • Page replacement algorithms • When a page fault occurs, which page should be removed to make room for the needed one? • NRU -- not recently used; FIFO; clock page; LRU • LRU is a good approximation to the optimal algorithm • Hard to implement in software • NFU -- not frequently used -- is a software way to simulate LRU ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Deja Vu all over again • Working set • notion regarding pages in interacting but distinct programs • If you can influence this -- improve performance? • Prefer an operating system that keeps track of which pages are in the working set. • When page fault occurs, evict a page not in the working set ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Design Issues for Paging Systems • We’ll come back to this slide in the week that we look at Performance patterns and anti-patterns. • Reminder: • Fast path “god” class • First Things First Excessive dynamic alloc • Coupling Circuitous Treasure Hunt • Batching One-Lane Bridge • Alternate Routes Traffic Jam • Flex Time • Slender Cyclic Functions ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

pdf handout from Tanenbaum Chapter 5 - Input/Output ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Principles of I/O Software • Device independence • “... should be possible to write programs that can access any I/O device without having to specify the device in advance” • “A program that reads a file as input should be able to read a file on a floppy disk, on a hard disk, or on a CD-ROM, without having to modify the program for each different device.” • sort <input >output should work with input coming from various types of disks or the keyboard and the output going to any disk or the screen ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

What’s that got to do with multi-program systems? • It is up to the o.s. to take care of the problems caused by the fact that these devices really are different and require different command sequences to read or write. • You may be responsible for that in the absence of an o.s. or with an inadequate one for the embedded system But • In multi-program communication, the data flow can be between programs and many of the same techniques can be used. ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Fill in for device I/O managed by CPU - 5.2 ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Fill in analogous concept for inter-program communication ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

I/O Software System Layers User-level I/O software Device-independent operating system software Device drivers Interrupt handlers Hardware ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

How might a program communicate with another? • Driver starting an I/O operation blocks until the I/O has completed and the interrupt occurs. • driver blocks itself by setting a semaphore, a wait on a condition variable, a receive on a message or something similar • When the interrupt happens, the procedure • does whatever it has to do in order to handle the interrupt • then unblocks the driver (reset the semaphore, execute an interrupt return to the previous process) ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Role for “device” driver • An exercise in layered software architecture • Standard interface procedures that the rest of the o.s. can call • Functions: • accept abstract read and write requests from the device-independent software and see that they are carried out • initialize the device • manage power requirements and log events • check input parameters to see if valid (Meyer: Filter-methods in Design by Contract programming) • may need to translate parameters from abstract to concrete terms • check if device is currently in use; if so, queue the request • list continued on next slide ... ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

continued • timing: check if request can be handled now or if something else must be done like start the device • control the device -- issue sequence of commands, may need to interact sequentially • driver blocks itself until interrupt indicates completion or it doesn’t need to • check for errors • may pass data • returns some status information for error reporting • Select and start another request or blocks, waiting for next request • must be reentrant • cannot make system calls; may need to make certain kernel calls ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Think of the device driver as a shared resource .. • Look in Modern Operating Systems, Chapter 2 on Processes and Threads, Section 2.3 on Interprocess Communication • Which of these forms of communication does a program typically use with a device driver? • Why are some techniques in Section 2.3 appropriate for program-program communication but not appropriate (or not necessary) for program-device-driver communication? • Analyze the interprocess comm techniques with respect to their suitability to accomplish “device driver” functions in program-program communication. ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

User Space I/O Software -- (parallels to inter-program communication?) • Library procedures • Spooling • A way of dealing with dedicated I/O devices in a multiprogramming system • special process, daemon • special directory, spooling directory • process generates entire file to be output • process puts it in the spooling directory • the daemon extracts files from that directory and uses them as data ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

Can spooling be useful in multi-program interactions? ECEN5043 SW Eng of Multi-Program Systems, University of Colorado, Boulder

More Multiprogramming Issues