CENG334 Introduction to Operating Systems

1. Week 1 Erol Sahin Dept of Computer Eng. Middle East Technical University Ankara, TURKEY URL: http://kovan.ceng.metu.edu.tr/~erol/Courses/CENG334 Introduction Topics • What’s an operating system? • Course policy CENG334Introduction to Operating Systems Some of the following slides are adapted from Matt Welsh, Harvard Univ.

2. Welcome to CENG334! What is this course about? Operating Systems drive the inner workings ofvirtually every computer in the world today PCs, servers, iPods, cell phones, missile guidance systems,etc. all have an OS that dictate how they operate. The OS manages many aspects of how programs run, and howthey interact with hardware and the outside world.

4. Welcome to CENG334! What is this course about? Operating Systems drive the inner workings ofvirtually every computer in the world today PCs, servers, iPods, cell phones, missile guidance systems,etc. all have an OS that dictate how they operate. The OS manages many aspects of how programs run, and howthey interact with hardware and the outside world. Understanding the OS is essential for understanding: System performance and reliability Resource management Virtualization and abstraction Concurrency and parallelism Hardware interfaces and I/O This course is about more than just “kernel internals” It is really about learning complex systems design.

5. What is an operating system? User application User application User application Protection boundary Kernel Memory management Process management Filesystem TCP/IP stack Accounting CPU support Device drivers Disk I/O Hardware/software interface Gnarly world of hardware Software that provides an elaborate illusion to applications

6. One OS Function: Concurrency I think I have my own CPU So do I! Give every application the illusion of having its own CPU!

7. One OS Function: Concurrency Kernel Scheduler time • The OS timeslices each application on a single CPU • Switches between applications extremely rapidly, i.e., 100 times/sec Timeslice on single CPU system

8. Another OS Function: Virtual Memory Physical RAM Code Code Data Data Stack Stack VM System Code Code Data Data Stack Stack Swap out to disk • Give every application the illusion of having infinite memory • And, that it can access any memory address it likes! • In reality, RAM is split across multiple applications

9. More OS Functions • Multiprocessor support • Modern systems have multiple CPUs • Can run multiple applications (or threads within applications) in parallel • OS must ensure that memory and cache contents are consistent across CPUs • Filesystems • Real disks have a hairy, sector-based access model • User applications see flat files arranged in a hierarchical namespace • Network protocols • Network interface hardware operates on the level of unreliable packets • User apps see a (potentially reliable) byte-stream socket • Security and protection • Prevent multiple apps from interfering with each other and with normal system operation

10. Why bother with an OS? • Not just to give Slashdot readers something to argue about... • What do you think?

11. Why bother with an OS? • Not just to give Slashdot readers something to argue about... • Abstract away messy details of hardware • Give apps a nice clean view of the system • Save programmers a lot of trouble when building applications • Allow apps to be ported across a wide range of hardware platforms • Safety! • Don't let applications run amok – keep them in a “sandbox” • e.g., Access to unallocated memory address crashes only the program, not the whole system • Segmentation fault – core dumped • Efficiency • Share one machine across many different apps: concurrent execution • You would be surprised how much slack there is in a typical computer system

12. Why study operating systems? Most people will never write one from scratch... (Though if you do you stand to get incredibly rich)‏ Although more and more people are hacking them (e.g., Linux and BSD)‏ You need to understand the “big picture” in order to hack the details This class is about much more than the kernel! Data structures, concurrency, performance, resource management, synchronization, networks, distributed systems, databases ... The ideas and skills you pick up in this class have broad applications And it doesn't hurt for those Microsoft interviews either This course is the basis for future work in other areas of systems Distributed systems, P2P, sensor nets, etc.

13. Major OS Design Issues • Structure • How is the OS itself organized? Lots of modules? One big blob of code? • Sharing • How are limited resources multiplexed across users? • Naming • How to programs and users name and access resources? • Security • How to prevent malicious users from compromising the system? • Performance • How to keep it all fast? • Reliability • What happens when a program (or the OS itself) has a bug or failure?

14. More OS Design Issues • Extensibility • How do users and developers add new features? • Communication • How do programs exchange information? • Concurrency • How are multiple concurrent activities created and controlled? • Scale • What happens when demands on resources increase? • Distribution • How do many computers interact with each other, e.g., to pool resources? • Accounting • How do you track (and maybe charge for) resource usage?

15. Teaching staff Instructor: Section 1, 2: Dr. Erol Sahin Location: B-111, Tel: 210 5539, E-mail: erol@ceng Office hours: by appointment. Teaching assistant: Can Hosgor hosgor@ceng Erdal Sivri E-mail: erdal@ceng

16. Textbook Operating System Concepts Avi Silberschatz Peter Baer Galvin Greg Gagne John Wiley & Sons, Inc. ISBN 0-471-76907-X Modern Operating Systems Andrew S. Tanenbaum Prentice Hall., 2001, ISBN-10: 0130313580 ISBN-13: 97801303135 Both books should be available at the bookstore..

17. Grading Midterm: 30 % [April 14] Assignments: 30 % Final: 40 %

18. Assignments Assignment 1: Basic Shell process management, Unix system calls Assignment 2: Basic Threading thread management and synchronization Assignment 3: File System ext2 file system implementation, inode/block concepts, file/directory handling

19. Policies Late assignments: Late submission policy will be announced for each assignment. Academic dishonesty: All assignments submitted should be fully your own. We have a zero tolerance policy on cheating and plagiarism. Your work will be regularly checked for such misconduct and any such attempts will be prosecuted: At all times you have the right to challenge our decisions on cheating, upon which the case will be processed through the disciplinary channels of the university. However, we would like to remind you that, if found guilty, the legal code of the university proposes a minimum of six month expulsion from the university.

20. Cheating What is cheating? Sharing code: either by copying, retyping, looking at, or supplying a copy of a file. What is NOT cheating? Helping others use systems or tools. Helping others with high-level design issues. Helping others debug their code.

21. Communication Online information about the course will be available on the CENG334 web page: http://kovan.ceng.metu.edu.tr/~erol/Courses/CENG334/ Announcements about the course will be made at the CENG334 newsgroup at news://metu.ceng.course.334 Please put Section 1,2 on the subject line of your posting. If you have a specific question you can send an e-mail to the us. However make sure that the subject line starts with CENG334 [capital letters, and no spaces] to get faster reply.

22. Erol Sahin Dept of Computer Eng. Middle East Technical University Ankara, TURKEY URL: http://kovan.ceng.metu.edu.tr/~erol/Courses/CENG334 Operating System Overview Topics • Brief History • OS Services • System calls • Basic Operation • OS structures CENG334Introduction to Operating Systems Some of the following slides are adapted from Matt Welsh, Harvard Univ.

23. In the Beginning... • There was no OS – just libraries • Computer only ran one program at a time, so no need for an OS • Programming through wiring.. ENIAC, 1945 Harvard Mark I, 1944 IBM 360, 1960's

24. In the Beginning... • There was no OS – just libraries • Computer only ran one program at a time, so no need for an OS • And then there were batch systems • Programs printed on stacks of punchhole cards • OS was resident in a portion of machine memory • When previous program was finished, OS loaded next program to run

25. Punch Card

26. In the Beginning... • There was no OS – just libraries • Computer only ran one program at a time, so no need for an OS • And then there were batch systems • Programs printed on stacks of punchhole cards • OS was resident in a portion of machine memory • When previous program was finished, OS loaded next program to run • Disk spooling • Disks were much read stack onto disk while previous program is running • With multiple programs on disk, need to decide which to run next! • But, CPU still idle while program accesses a peripheral (e.g., tape or disk!)‏

27. Multiprogramming • To increase system utilization, multiprogramming OS’s were invented • keeps multiple runnable jobs loaded in memory at once • Overlaps I/O of a job with computing of another • While one job waits for I/O to compile, CPU runs instructions from another job • To benefit, need asynchronous I/O devices • need some way to know when devices are done performing I/O • Goal: optimize system throughput • perhaps at the cost of response time… Dennis Ritchie and Ken Thompson at a PDP11, 1971

28. Timesharing • To support interactive use, timesharing OS's were created • multiple terminals connected to one machine • each user has illusion of entire machine to him/herself • optimize response time, perhaps at the cost of throughput • Timeslicing • divide CPU fairly among the users • if job is truly interactive (e.g. editor), then can switch between programs and users faster than users can generate load • MIT Multics (mid-1960’s) was the first large timeshared system • nearly all modern OS concepts can be traced back to Multics

29. Personal Computing Apple I, 1976 • Totally changed the computing industry. • CP/M: First personal computer OS • IBM needed OS for their PCs, CP/M behind schedule • Bill Gates to the rescue: Bought 86-DOS and made MS-DOS • DOS is basically a subroutine library! • Many popular personal computers follow • Apple, Commodore, TRS-80, TI 99/4, Atari, etc... Apple LISA, 1983 IBM PC, 1981 Bill Gates and Paul Allen, c.1975 Commodore VIC-20

31. Parallel Computing and Clusters • High-end scientific apps want to use many CPUs at once • Parallel processing to crunch on enormous data sets • Need OS and language primitives for dividing program into parallel activities • Need OS primitives for fast communication between processors • degree of speedup dictated by communication/computation ratio • Many kinds of parallel machines: • SMPs: symmetric multiprocessors – several CPUs accessing the same memory • MPPs: massively parallel processors – each CPU may have its own memory • Clusters: connect a lot of commodity machines with a fast network

32. Distributed OS • Goal – Make use of geographically distributed resources • workstations on a LAN • servers across the Internet • Supports communication between applications • interprocess communication (on a single machine): • message passing and shared memory • networking procotols (across multiple machines): • TCP/IP, Java RMI, .NET SOAP • “The Grid”, .NET, and OGSA • Idea: Seamlessly connect vast computational resources across the Internet

33. Embedded OS • The rise of tiny computers everywhere – ubiquitous computing • Processor cost low enough to embed in many devices • PDAs, cell phones, pagers, ... • How many CPUs are in your car? On your body right now? • Gets more interesting with ubiquitous networking! • Wireless networks becoming pervasive • Sensor networks are an exciting new direction here • Little “motes” with less 4KB of RAM, some sensors, and a radio • Typically very constrained hardware resources • slow processors • very small amount of memory (e.g. 8 MB)‏ • no disk – but maybe quasi-permanent storage such as EEPROM

34. Operating System Overview User application User application User application Protection boundary Kernel Memory management Process management Filesystem TCP/IP stack Accounting CPU support Device drivers Disk I/O Hardware/software interface

35. Operating System Services(What things does the OS do?)‏ • Services that (more-or-less) map onto components • Program execution • How do you execute concurrent sequences of instructions? • I/O operations • Standardized interfaces to extremely diverse devices • File system manipulation • How do you read/write/preserve files? • Looming concern: How do you even find files??? • Communications • Networking protocols/Interface with CyberSpace? • User interface- Almost all operating systems have a user interface (UI)‏ • Varies between Command-Line (CLI), Graphics User Interface (GUI), Batch • Cross-cutting capabilities • Error detection & recovery • Resource allocation • Accounting • Protection

36. User Operating System Interface - CLI • CLI allows direct command entry • Sometimes implemented in kernel, sometimes by systems programs • Sometimes multiple flavors implemented – shells • Primarily fetches a command from user and executes it • Sometimes commands built-in, sometimes just names of programs • If the latter, adding new features doesn’t require shell modification

37. User Operating System Interface - GUI • User-friendly desktop metaphor interface • Usually mouse, keyboard, and monitor • Icons represent files, programs, actions, etc • Various mouse buttons over objects in the interface cause various actions • provide information, options, • execute function, open directory (known as a folder) • Invented at Xerox PARC • Many systems now include both CLI and GUI interfaces • Microsoft Windows is GUI with CLI “command” shell • Apple Mac OS X as “Aqua” GUI interface with UNIX kernel underneath and shells available • Solaris is CLI with optional GUI interfaces (Java Desktop, KDE)

38. Xerox PARC Alto

39. System Calls • Programming interface to the services provided by the OS • Typically written in a high-level language (C or C++) • Mostly accessed by programs via a high-level Application Program Interface (API) rather than direct system call use • Three most common APIs are • Win32 API for Windows, • POSIX API for POSIX-based systems (including virtually all versions of UNIX, Linux, and Mac OS X), and • Java API for the Java virtual machine (JVM) • Why use APIs rather than system calls?

40. Example of Standard API • Consider the ReadFile() function in the • Win32 API—a function for reading from a file • A description of the parameters passed to ReadFile()‏ • HANDLE file—the file to be read • LPVOID buffer—a buffer where the data will be read into and written from • DWORD bytesToRead—the number of bytes to be read into the buffer • LPDWORD bytesRead—the number of bytes read during the last read • LPOVERLAPPED ovl—indicates if overlapped I/O is being used

41. System Call Implementation • Typically, a number associated with each system call • System-call interface maintains a table indexed according to these numbers • The system call interface invokes intended system call in OS kernel and returns status of the system call and any return values • The caller need know nothing about how the system call is implemented • Just needs to obey API and understand what OS will do as a result call • Most details of OS interface hidden from programmer by API Managed by run-time support library (set of functions built into libraries included with compiler)

42. API – System Call – OS Relationship

43. System Programs • System programs provide a convenient environment for program development and execution. They can be divided into: • File manipulation • Status information • File modification • Programming language support • Program loading and execution • Communications • Application programs • Most users’ view of the operation system is defined by system programs, not the actual system calls

44. System Programs • Provide a convenient environment for program development and execution • Some of them are simply user interfaces to system calls; others are considerably more complex • File management - Create, delete, copy, rename, print, dump, list, and generally manipulate files and directories • Status information • Some ask the system for info - date, time, amount of available memory, disk space, number of users • Others provide detailed performance, logging, and debugging information • Typically, these programs format and print the output to the terminal or other output devices • Some systems implement a registry - used to store and retrieve configuration information

45. System Programs (cont’d)‏ • File modification • Text editors to create and modify files • Special commands to search contents of files or perform transformations of the text • Programming-language support - Compilers, assemblers, debuggers and interpreters sometimes provided • Program loading and execution- Absolute loaders, relocatable loaders, linkage editors, and overlay-loaders, debugging systems for higher-level and machine language • Communications - Provide the mechanism for creating virtual connections among processes, users, and computer systems • Allow users to send messages to one another’s screens, browse web pages, send electronic-mail messages, log in remotely, transfer files from one machine to another

46. Memory Emacs Firefox xmms sshd Operating System operation • The OS kernel is just a bunch of code that sits around in memory, waiting to be executed OS Kernel (device drivers, file systems, virtual memory, etc.)‏

47. Interrupt (disk block read)‏ System call (open network socket)‏ Operating System operation • The OS kernel is just a bunch of code that sits around in memory, waiting to be executed • OS is triggered in two ways: system calls and hardware interrupts • System call: Direct “call” from a user program • For example, open() to open a file, or exec() to run a new program • Hardware interrupt: Trigger from some hardware device • For example, when a disk block has been read or written Memory Emacs OS Kernel (device drivers, file systems, virtual memory, etc.)‏ Firefox xmms sshd

48. Interrupts – a primer • An interrupt is a signal that causes the CPU to jump to a pre-defined instruction – called the interrupt handler • Interrupt can be caused by hardware or software • Hardware interrupt examples • Timer interrupt (periodic “tick” from a programmable timer) • Device interrupts • e.g., Disk will interrupt the CPU when an I/O operation has completed • Software interrupt examples (also called exceptions) • Division by zero error • Access to a bad memory address • Intentional software interrupt – e.g., x86 “INT” instruction • Can be used to trap from user program into the OS kernel! • Why might this be useful?

49. Interrupt handler for interrupt 4 Interrupt handler for interrupt 5 !!! Interrupt handler example 1) OS fills in interrupt handlertable (usually at boot time)‏ Interrupt handler table 2) Interrupt occurs – e.g., hardwaresignal 3) CPU state saved to stack

50. Interrupt handler for interrupt 4 Interrupt handler for interrupt 4 Interrupt handler for interrupt 5 Interrupt handler example 1) OS fills in interrupt handlertable (usually at boot time)‏ Interrupt handler table 2) Interrupt occurs – e.g., hardwaresignal !!! 3) CPU state saved to stack 4) CPU consults interrupt table and invokes appropriate handler