CS 416 Operating Systems Design Spring 2008

1. CS 416Operating Systems DesignSpring 2008 Liviu Iftode iftode@cs.rutgers.edu

2. Course Overview Goals • Understand how an operating system works • Learn how OS concepts are implemented in a real operating system • Introduce to systems programming • Learn about performance evaluation • Learn about current trends in OS research

3. OS Class Framework Project OS Implementation OS Concepts (source code, project doc) (lectures, textbooks) recitations OS Programming Homework Real OS (man pages) (Unix/Linux textbooks)

4. Suggested Approach • Read the assigned chapter from the textbook before the lecture to understand the basic idea and become familiar with the terminology • Attend the recitation • Start homework and project right away, systems programming is not easy, it takes a lot of time ! • Ask questions during lecture, recitation. • Use the mailing list/newsgroup/forum for discussions, do not be afraid to answer a question posted by your colleague even if you are not sure. This is a way to validate your understanding of the material. Do not forget, questions and discussions are not graded !

5. Course Outline • Processes and Process Management • Threads and Thread Programming • Synchronization and Deadlock • Memory Management and Virtual Memory • CPU Scheduling • File Systems and I/O Management • Networking and Distributed Systems • Security

6. Course Requirements Prerequisites • Computer Architecture • Good C programming skills Expected work • Substantial readings (textbooks and papers) • Challenging project • Homework (requires programming) • Midterm and final exams

7. Work Evaluation • Midterm exam 30% • Homework + Project 40% • Final exam 30%

8. Homework Goals • Deepen the understanding of OS concepts • Develop systems programming skills: virtual memory, threads, synchronization, sockets • Learn to design, implement, debug and evaluate the performance of an OS-bound program Structure • 4-5 assignments • Both theoretical and C-programming problems

9. Project Goals • Learn to design, implement and evaluate basic OS mechanisms and policies Structure • Individual + team project • Multiple phases: some individual, some team work • Project report for each phase • Possibly, short oral presentation of final report

10. Textbooks • Silberschatz, Galvin and Gagne, Operating System Concepts, 7th Edition, John Wiley & Sons, 2004. • Stallings Operating Systems. Internals and Design Principles, 5th Edition, Prentice-Hall, 2005. • Papers will be made available on the course homepage • Project will require additional textbooks

11. Logistics • TAs • Brian Russell (morbius@cs) • Steve Smaldone (smaldone@cs) • Course homepage: http://www.cs.rutgers.edu/~iftode/cs416_2008.html • Preliminary schedule and lecture will be made available for the entire semester but they may be updated before each class. • Homework or project every other week. • A mailing list/newsgroup/forum will be announced shortly. • Check course homepage for announcements

12. Basic Computer Structure Memory CPU memory bus I/O bus disk Net interface

13. Computer System • Processor: performs data processing • Main memory: stores both data and programs, typically volatile • Disks: secondary memory devices which provide persistent storage • Network interfaces: inter-machine communication • Buses: intra-machine communication • memory bus (processor-memory) • I/O bus (disks, network interfaces, other I/O devices, memory-bus)

14. What is an Operating System • a software layer between the hardware and the application programs/users that provides a virtual machine interface: easy and safe • a resource manager that allows programs/users to share the hardware resources: fair and efficient application (user) operating system hardware

15. How Does an OS Work? • receives requests from the application: system calls • process the requests: may issue commands to hardware • handles hardware interrupts: may upcall the application • OS complexity: synchronous calls + asynchronous events application (user) system calls upcalls hardware independent OS commands interrupts hardware dependent hardware

16. Mechanism vs. Policy • mechanism: data structures and operations that implement the abstraction (e.g. the buffer cache) • policy: the procedure that guides the selection of a certain course of action from among alternatives (e.g. the replacement policy for the buffer cache) • traditional OS is rigid: mechanism together with policy application (user) operating system: mechanism+policy hardware

17. Mechanism-Policy Split • OS cannot provide the best policy in all cases • new OS approaches separate mechanisms from policies: • OS provides the mechanism + some policy • applications contribute to the policy • flexibility+efficiency require new OS structures and/or new OS interfaces

18. Processes A process is a processor abstraction • traditional approach: OS switches processes on the processor (process scheduling), provides inter-process communication and handles exceptions • new approaches: application-controlled scheduling, reservation-based scheduling, agile applications user: run application create, kill processes, inter-process comm operating system: processes context switch hardware: processor

19. Traditional Process Scheduling active • OS schedules processes on the processor • OS mediates inter-process communication (IPC) • Application provides hints: priorities processes IPC OS processor

20. Hierarchical Scheduling • OS schedules schedulers which schedule dependent processes schedulers processes OS processor

21. Virtual Memory Virtual memory is a memory abstraction • traditional approach: OS provides a sufficiently large virtual address space for each running application, does memory allocation and replacement and may ensure protection • new approaches: external memory management, huge (64-bit) address space, global memory application: address space virtual addresses operating system: virtual memory physical addresses hardware: physical memory

22. Virtual Memory: Mechanism vs. Policy virtual address spaces p1 • processes can run being partially loaded in memory • illusion of more memory than physically available: swapping • processes can share physical memory if permitted • replacement policy can be exposed to the application p2 processes: v-to-p memory mappings physical memory:

23. Files A file is a storage abstraction • traditional approach: OS does disk block allocation and caching (buffer cache) , disk operation scheduling and replacement in the buffer cache • new approaches: application-controlled cache replacement, log-based allocation (makes writes fast) application/user: copy file1 file2 naming, protection, operations on files operating system: files, directories operations on disk blocks... hardware: disk

24. Traditional File System application: read/write files translate file to disk blocks OS: ... maintains ...buffer cache controls disk accesses: read/write blocks hardware:

25. Application-Controlled Caching application: read/write files replacement policy translate file to disk blocks OS: ... ...buffer cache maintains controls disk accesses: read/write blocks hardware:

26. Communication Message passing is a communication abstraction • traditional approach: OS provides naming schemes, reliable transport of messages, packet routing to destination • new approaches: zero-copy protocols, active messages, memory-mapped communication application: sockets naming, messages operating system: TCP/IP protocols network packets hardware: network interface

27. Traditional OS structure • monolithic/layered systems • one/N layers all executed in “kernel-mode” • good performance but rigid user process user system calls memory system file system OS kernel hardware

28. Micro-Kernel OS memory server client process file server user mode IPC micro-kernel hardware • client-server model, IPC between clients and servers • the micro-kernel provides protected communication • OS functions implemented as user-level servers • flexible but efficiency is the problem • easy to extend for distributed systems

29. Extensible OS Kernel process process user mode my memory service default memory service extensible kernel hardware • user processes can load customized OS services into the kernel • good performance but protection and scalability become problems

30. Virtual Machines • the real hardware in “cloned” in several identical virtual machines • OS functionality built on top of the virtual machine applications applications OS-2 on virtual machine 2 OS-1 on virtual machine 1 Virtual Machine Monitor Hardware

31. Next Time • Computer Architecture Refresher • Silberschatz, Chapter 1 • Stallings, Chapter 1