Having fun withy the Unix Operating System

1. Having fun withy the Unix Operating System Praxis Week 7 Rob Pooley System Level Programming Operating Systems

2. What is the OS? • The operating system provides a layer between the programmer and the underlying system architecture, giving higher level instructions ("system calls"), and insulating the programmer from having to think about low level issues. • The OS enables processor resources and memory to be shared among many programs running at the same time. System Level Programming Operating Systems

3. We'll focus on three things: • Memory: How is the programmer insulated from having to think about real physical memory limitations? • I/O: How does the OS support the programmer in managing I/O to various peripheral devices? • Processes: How can several programs (processes) run at the same time (without the programmer having to worry about how). System Level Programming Operating Systems

4. Virtual Memory • Computer memory consists of • primary memory - RAM (random access memory) • various forms of secondary (disk) storage. • RAM is so called because you can access the contents of a memory location directly if you know the location. • So, ideally we want our program and data in RAM. • But what happens if we want to run a lot of large programs, and there is not enough RAM? System Level Programming Operating Systems

5. The solution (now almost universally adopted) is to use "virtual memory". • Virtual memory separates the concepts of address space and actual memory locations. • We can see how this works by considering computers with a very small amount of memory – le’ts say 4096 bytes. • If the processor used 2 byte integers to hold addresses it could in principle refer to 65536 (2 ** 16) locations. • But only 4096 of these could actually be locations in physical RAM. • If a program or programs requires more than these 4096 bytes to hold its code and data segments, some of this must be copied to disk. System Level Programming Operating Systems

6. With virtual memory the OS is responsible for managing a mapping between • the address space used in the programs (i.e., the addresses used in the stack etc) and • the actual physical RAM (and disk). • The computer can look at the RAM for areas that have not been used recently and copy them onto hard disk. • This frees up space for new programs to run. • As this mapping and copying is done automatically within the operating system the programmer (high level or assembly) can use as much of the full address space as will fit onto the disc and not be limited to the physical RAM. System Level Programming Operating Systems

7. 2 pages mapped into memory Memory has a free page Data needed from backing store System Level Programming Operating Systems

8. Third page swapped into memoryA page fault Physical memory full System Level Programming Operating Systems

9. Another page needed from discA page fault Physical memory full More data needed from backing store System Level Programming Operating Systems

10. One page written back onto disc Physical memory has a free page System Level Programming Operating Systems

11. New page fetched into memory Physical memory full again Need to write back before fetching System Level Programming Operating Systems

12. Input and Output • Input and output refers not just to the stuff that you read in and write out when you run your program, but also to interaction with peripheral devices (printers etc). • Fortunately we can use the same model for both: writing something out on the user's screen, writing to a file, and writing to a printer all are based on the same principles. • The operating system handles the low level details of how devices are actually written to. System Level Programming Operating Systems

13. Files and I/O Devices • Input and output is based on the notion of a "file". • But a file doesn’t just refer to something you store on your hard disk, but to any sequence of bytes that can be written to an i/o device. • So we can write a file to a printer (or any other output device, such as an audio device). • We can read from our floppy drive, hard drive, CD player etc. • The programmer does not have to be concerned with how to "activate" these devices and locate files. • The physical organisation of files on (say) the hard disk is quite separate from the logical file structure that the programmer or user uses to refer to them. System Level Programming Operating Systems

14. I/O system calls • The operating system provides system calls for opening, reading, writing and closing files. • When you open a file in "C" (or Java) you will be invoking the appropriate operating system call. • Opening a file involves locating it, and bringing into memory information necessary to access it. • Reading it involves accessing the file and the first part of the data it contains. • As we will usually not be reading all the data in the buffer at once, a pointer is updated indicating which byte should be read next. System Level Programming Operating Systems

15. Buffering • In practice I/O is usually buffered. • We don't read/write directly to the physical device, but to a temporary storage area (the buffer). • The OS handles the interface between this buffer and the actual physical device, transforming the data as required and reading/writing fixed sized chunks for efficiency. • This is referred to as stream I/O. • The user/programmer can usually ignore this, except abnormal cases where the buffer written to fails to be flushed/written out. System Level Programming Operating Systems

16. Buffering System Level Programming Operating Systems

17. Directories • Although the hard bit of file handling is left to the operating system, the user/programmer needs a way • to refer to files (names) • to keep them organised (folders or directories). • Directories provide • a logical structure for users to keep their files organised, • a structure suitable for adding security instructions to prevent unauthorised use • you can change permissions on a directory so that only you (or your "group") can read or write to files within it. • The operating system provides system calls for managing this structure (e.g., creating a directory; moving a file into it) and altering permissions. System Level Programming Operating Systems

18. I/O devices • The directory structure also provides a means for specifying input/output devices. • On Unix the "/dev" directory contains files (try ls /dev) but the files there correspond to devices, not to files on the hard disk. • If you write to these files the necessary device driver will be invoked by the operating system - this is a bit of software that knows how to start up, read and write to this particular device. System Level Programming Operating Systems

19. You can get more information about devices simply using ls -l(ie, a detailed file listing): pele% ls -l /dev/tty crw-rw-rw- 1 root root 5, 0 Aug 22 09:34 /dev/tty pele% ls -l /dev/fd0 brw-rw-rw- 1 ceebde1 floppy 2, 0 May 5 1998 /dev/fd0 pele% ls -l /dev/lp0 crw-rw---- 1 root daemon 6, 0 May 5 1998 /dev/lp0 pele% ls -l /dev/audio crw-rw-rw- 1 ceebde1 bin 14, 4 May 5 1998 /dev/audio • In the first example, the very first letter (c) indicates that it is a character type device. • The numbers (5, 2, 6, 14) are part of what is used to indicate the device driver to use. System Level Programming Operating Systems

20. You can write to these devices just as you would write to files. • But you have to know what sort of data it will make sense to write. • Printers may expect postscript files; audio devices will expect particular audio data formats. • Normally your floppy (or CD writer) is used to hold files of particular types, not just streams of characters. • We will therefore generally restrict examples to writing to the terminal (the device /dev/tty) or to the hard disk. System Level Programming Operating Systems

21. Unix I/O • Normally Unix programs read input from what is called the "standard input stream" and write to the "standard output stream". • By default, standard input will be the keyboard, and standard output the terminal (screen). that is, the place to read and write stuff if no other file is specified. • By default the standard input and output is your terminal - or the device /dev/tty. • But it is possible (and easy) to redirect standard input and output, so that any device or file is used for i/o. System Level Programming Operating Systems

22. Redirecting I/O • To redirect standard output the ">" symbol is used. • If we use the command "ls" it outputs to the standard output, the terminal. But we can redirect this to a file: %pele ls > myfile • Try this, and look at the contents of the file. If you then want to add more on to the end of the file we can use ">>": %pele date >> myfile • This will stick today’s date at the end of your file, after your directory listing System Level Programming Operating Systems

23. To append redirected output, rather than overwriting the file use “>>” • To redirect standard input use the "<" symbol. If a program usually takes input from the terminal, you can get it to take the input from a file. • You could, for example, mail the contents of a file to someone using: mail alison < myfile System Level Programming Operating Systems

24. Redirecting other devices • In general either < or > can be preceded by the number of a file descriptor, forcing redirection of that device. cat fred >jim 2>alice Redirects stdout to jim and stderr to alice • stdin is number 0 and stdout is number 1, but these can usually be omitted System Level Programming Operating Systems

25. Pipes • You can also arrange for the input for one program to come from another program. This allows us to string together a sequence of simple commands. • Pipes ("|") are used for this. They take the output stream of one program and connect it to the input stream of another. • Suppose we want our friends to know what's in our directory. We could use: ls | mail fred • The output of "ls" becomes the input of "mail" and a mail message is sent containing the directory listing. Try using this approach to mail yourself today’s date. System Level Programming Operating Systems