Advanced Operating Systems

1. Advanced Operating Systems • Course Reading Material: • Lecture notes, class discussions will be the material for examination purposes. • Recommend reading appropriate reference papers for the different algorithms. • (May be out of print): “Advanced Concepts in Operating Systems: Distributed, Database, & Multiprocessor Operating Systems” by Mukesh Singhal and Niranjan G. Shivaratri. McGraw Hill publishers. • Other books discussing the algorithms will help too. B. Prabhakaran

2. Advanced Operating Systems • Project Reference Text: • “Unix Network programming”, Richard Stevens, Prentice Hall. B. Prabhakaran

3. Contact Information B. Prabhakaran Department of Computer Science University of Texas at Dallas Mail Station EC 31, PO Box 830688 Richardson, TX 75083 Email: praba@utdallas.edu Fax: 972 883 2349 URL: http://www.utdallas.edu/~praba/cs6378.html Phone: 972 883 4680 Office: ES 3.706 Office Hours: 10-11am, Mondays & Wednesdays Other times by appointments through email Announcements: Made in class and on course web page. TA: TBA. B. Prabhakaran

4. Course Outline Proposed Outline. Might be modified based on time availability: • Introduction to operating systems, inter-process communication. (Chapters 1 & 2). • Distributed Operating Systems • Architecture (Chapter 4) • Clock Synchronization, Ordering (Chapter 5) • Distributed Mutual Exclusion (Chapter 6) • Distributed Deadlock Detection (Chapter 7) • Agreement Protocols (Chapter 8) • Distributed Resource Management • Distributed File Systems (Chapter 9) • Distributed Shared Memory (Chapter 10) • Distributed Scheduling (Chapter 11) B. Prabhakaran

5. Course Outline ... • Recovery & Fault Tolerance • Chapters 12 and 13 • Concurrency Control/ Security • Depending on time availability Discussions will generally follow the main text. However, additional/modified topics might be introduced from other texts and/or papers. References to those materials will be given at appropriate time. B. Prabhakaran

6. Evaluation • 2 Mid-terms: in class. 75 minutes. Mix of MCQs (Multiple Choice Questions) & Short Questions. • 1 Final Exam: 75 minutes or 2 hours (Mix of MCQs and Short Questions). • 1 Quiz: in class. 5-6 MCQs or very short questions. 15-20 minutes each. • Homeworks/assignments: at most 2 • Programming Projects: • 1 preparatory (to warm up on thread and socket programming) • 1 Long project with an intermediate and final submission. B. Prabhakaran

7. Grading • 2 Home works: 5% • 1 Quiz: 5% • 2 Mid-terms + Final: 60% • Equally distributed 20% each • Preparatory Project: 5% • Intermediate & Final Projects (combined): 25% • Possible Letter Grades: A, A-, B+, B, B-, C+, C • Likely: B will be around class average • Letter grade trend will be informed after 2nd mid-term. B. Prabhakaran

8. Schedule • Quiz: September 17, 2008 • 1st Mid-term: October 6, 2008 • 2nd Mid-term: November 10, 2008 • Final Exam: December 15, 2008 (As per UTD schedule) • Subject to minor changes • Homework schedules will be announced in class and course web page, giving sufficient time for submission. • Likely project deadlines: • Preparatory project: September 14, 2008 • Intermediate project: October 26, 2008 • Final Project: December 7, 2008 B. Prabhakaran

9. Programming Projects • No copying/sharing of code/results will be tolerated. Any instance of cheating in projects/homeworks/exams will be reported to the University. • No copying code from the Internet. • 2 individual students copying code from Internet independently: still considered copying in the project !! • Individual projects. • Deadlines will be strictly followed for projects and homeworks submissions. • Projects submissions through Web CT. • Demo will be needed B. Prabhakaran

10. Two-track Course • Programming Project Discussions: • Announcements in class • Minimal Discussions in class • Design discussions during office hours based on individual needs • Theory (Algorithms) Discussions: • Full discussion in class • Office hours for clarification if needed B. Prabhakaran

11. Web CT • Go to: http://webct6.utdallas.edu • Web CT has a discussion group that can be used for project and other course discussions. B. Prabhakaran

12. Cheating • Academic dishonesty will be taken seriously. • Cheating students will be handed over to Head/Dean for further action. • Remember: home works/projects (exams too !) are to be done individually. • Any kind of cheating in home works/ projects/ exams will be dealt with as per UTD guidelines. • Cheating in any stage of projects will result in 0 for the entire set of projects. B. Prabhakaran

13. Projects • Involves exercises such as ordering, deadlock detection, load balancing, message passing, and implementing distributed algorithms (e.g., for scheduling, etc.). • Platform: Linux/Windows, C/C++. Network programming will be needed. Multiple systems will be used. • Specific details and deadlines will be announced in class and course webpage. • Suggestion: Learn network socket programming and threads, if you do not know already. Try simple programs for file transfer, talk, etc. • Sample programs and tutorials available at: • http://www.utdallas.edu/~praba/projects.html B. Prabhakaran

14. Homeworks • At most 2 home works, announced in class and course web page. • Homeworks Submission: • Submit on paper to TA/Instructor. B. Prabhakaran

15. Basic Computer Organization • Input Unit • Output Unit • CPU • Memory • ALU (Arithmetic & Logic Unit) • Secondary Storage Disk Memory ALU CPU I/O Display Keyboard B. Prabhakaran

16. Simplified View of OS OS Kernel User i Process j Physical Memory OS Tools Code Code User Processes Code Code Data Data User Processes .. Data Virtual Memory Data Tools ++ Memory Space B. Prabhakaran

17. Distributed View of the System hardware hardware hardware Process hardware hardware B. Prabhakaran

18. Inter-Process Communication • Need for exchanging data/messages among processes belonging to the same or different group. • IPC Mechanisms: • Shared Memory: Designate and use some data/memory as shared. Use the shared memory to exchange data. • Requires facilities to control access to shared data. • Message Passing: Use “higher” level primitives to “send” and “receive” data. • Requires system support for sending and receiving messages. • Operation oriented language constructs • Request-response action • Similar to message passing with mandatory response • Can be implemented using shared memory too. B. Prabhakaran

19. IPC Examples • Parallel/distributed computation such as sorting: shared memory is more apt. • Using message passing/RPC might need an array/data manager of some sort. • Client-server type: message passing or RPC may suit better. • Shared memory may be useful, but the program is more clear with the other types of IPCs. • RPC vs. Message Passing: if response is not a must, atleast immediately, simple message passing should suffice. B. Prabhakaran

20. Shared Memory Writers/ Producers Only one process can write at any point in time. No access to readers. Shared Memory Multiple readers can access. No access to Writers. Readers/ Consumers B. Prabhakaran

21. Shared Memory: Possibilities • Locks (unlocks) • Semaphores • Monitors • Serializers • Path expressions B. Prabhakaran

22. Message Passing • Blocked Send/Receive: Both sending and receiving process get blocked till the message is completely received. Synchronous. • Unblocked Send/Receive: Both sender and receiver are not blocked. Asynchronous. • Unblocked Send/Blocked Receive: Sender is not blocked. Receiver waits till message is received. • Blocked Send/Unblocked Receive: Useful ? • Can be implemented using shared memory. Message passing: a language paradigm for human ease. B. Prabhakaran

23. Un/blocked • Blocked message exchange • Easy to: understand, implement, verify correctness • Less powerful, may be inefficient as sender/receiver might waste time waiting • Unblocked message exchange • More efficient, no waste on waiting • Needs queues, i.e., memory to store messages • Difficult to verify correctness of programs B. Prabhakaran

24. Message Passing: Possibilities receiver i receiver i sender receiver j sender receiver j receiver k receiver k B. Prabhakaran

25. Message Passing: Possibilities... sender i sender i receiver sender j receiver sender j sender k sender k B. Prabhakaran

26. Naming • Direct Naming • Specify explicitly the receiver process-id. • Simple but less powerful as it needs the sender/receiver to know the actual process-id to/from which a message is to be sent/received. • Not suitable for generic client-server models • Port Naming • receiver uses a single port for getting all messages, good for client-server. • more complex in terms of language structure, verification • Global Naming (mailbox) • suitable for client-server, difficult to implement on a distributed network. • complex for language structure and verification B. Prabhakaran

27. Communicating Sequential Processes (CSP) processreader-writer OKtoread, OKtowrite: integer (initially = value); busy: boolean (initially = 0); *[ busy = 0; writer?request() -> busy := 1; writer!OKtowrite;  busy = 0; reader?request() -> busy := 1; reader!OKtoread;  busy = 1; reader?readfin() -> busy := 0;  busy = 1; writer?writefn() -> busy := 0; ] B. Prabhakaran

28. CSP: Drawbacks • Requires explicit naming of processes in I/O commands. • No message buffering; input/output command gets blocked (or the guards become false) -> Can introduce delay and inefficiency. B. Prabhakaran

29. Operation oriented constructs Remote Procedure Call (RPC): Task A Task B Service declarations Service declarations send xyz; wait for result …… RPC: abc(xyz, ijk); ……. return ijk • Service declaration: describes in and out parameters • Can be implemented using message passing • Caller: gets blocked when RPC is invoked. • Callee implementation possibilities: • Can loop “accepting” calls • Can get “interrupted” on getting a call • Can fork a process/thread for calls B. Prabhakaran

30. RPC: Issues • Pointer passing, global variables passing can be difficult. • If processes on different machines, data size (number of bits for a data type) variations need to be addressed. • Abstract Data Types (ADTs) are generally used to take care of these variations. • ADTs are language like structures that specify how many bits are being used for integer, etc… • What does this imply? • Multiple processes can provide the same service? Naming needs to be solved. • Synchronous/blocked message passing is equivalent to RPC. B. Prabhakaran

31. Ada task proc-buffer is entrystore(x:buffer); remove(y:buffer); end; task [type] <name> is entry specifications end task body proc-buffer is temp: buffer; begin loop when flag accept store(x: buffer); temp := x; flag :=0; end store; When !flag accept remove(y:buffer); y := temp; flag :=1; end remove; end loop end proc-buffer. Task body <name> is Declaration of local variables begin list of statements ….. accept <entry id> (<formal parameters> do body of the accept statement end<entry id> exceptions Exception handlers end; B. Prabhakaran

32. Ada Message Passing Task A Task B entry store(...) xyz is sent from Task B to A …. accept Store(.) …... …. Store(xyz); ….. result Somewhat similar to executing procedure call. Parameter value for the entry procedure is supplied by the calling task. Value of Result, if any, is returned to the caller. B. Prabhakaran

33. RPC Design • Structure • Caller: local call + stub • Callee: stub + actual procedure • Binding • Where to execute? Name/address of the server that offers a service • Name server with inputs from service specifications of a task. • Parameter & results • Packing: Convert to remote machine format • Unpacking: Convert to local machine format B. Prabhakaran

34. RPC Execution Binding Server Register Services Receive Query Stub Local Proc. Stub Remote Proc. Local call Query binding server Return Server Address Execute procedure Unpack Params packing Local call Wait Pack results Unpack result Return Return Caller Callee B. Prabhakaran

35. RPC Semantics • At least once • A RPC results in zero or more invocation. • Partial call, i.e., unsuccessful call: zero, partial, one or more executions. • Exactly once • Only one call maximum • Unsuccessful? : zero, partial, or one execution • At most once • Zero or one. No partial executions. B. Prabhakaran

36. RPC Implementation • Sending/receiving parameters: • Use reliable communication? : • Use datagrams/unreliable? • Implies the choice of semantics: how many times a RPC may be invoked. B. Prabhakaran

37. RPC Disadvantage • Incremental results communication not possible: (e.g.,) response from a database cannot return first few matches immediately. Got to wait till all responses are decided. B. Prabhakaran

38. Distributed Operating Systems Issues : • Global Knowledge • Naming • Scalability • Compatibility • Process Synchronization • Resource Management • Security • Structuring • Client-Server Model B. Prabhakaran

39. DOS: Issues .. • Global Knowledge • Lack of global shared memory, global clock, unpredictable message delays • Lead to unpredictable global state, difficult to order events (A sends to B, C sends to D: may be related) • Naming • Need for a name service: to identify objects (files, databases), users, services (RPCs). • Replicated directories? : Updates may be a problem. • Need for name to (IP) address resolution. • Distributed directory: algorithms for update, search, ... B. Prabhakaran

40. DOS: Issues .. • Scalability • System requirements should (ideally) increase linearly with the number of computer systems • Includes: overheads for message exchange in algorithms used for file system updates, directory management... • Compatibility • Binary level: Processor instruction level compatibility • Execution level: same source code can be compiled and executed • Protocol level: Mechanisms for exchanging messages, information (e.g., directories) understandable. B. Prabhakaran

41. DOS: Issues .. • Process Synchronization • Distributed shared memory: difficult. • Resource Management • Data/object management: Handling migration of files, memory values. To achieve a transparent view of the distributed system. • Main issues: consistency, minimization of delays, .. • Security • Authentication and authorization B. Prabhakaran

42. DOS: Issues .. • Structuring • Monolithic Kernel: Not needed (e.g.,) file management not needed fully on diskless workstations. • Collective kernel: distributed functionality on all systems. • Micro kernel + set of OS processes • Micro kernel: functionality for task, memory, processor management. Runs on all systems. • OS processes: set of tools. Executed as needed. • Object-oriented system: services as objects. • Object types: process, directory, file, … • Operations on the objects: encapsulated data can be manipulated. B. Prabhakaran

43. DOS: Communication Computer Switch B. Prabhakaran

44. ISO-OSI Reference Model Application Application Presentation Presentation Session Session Communication Network Transport Transport Network Network Network Network Datalink Datalink Datalink Datalink Physical Physical Physical Physical B. Prabhakaran

45. Un/reliable Communication • Reliable Communication • Virtual circuit: one path between sender & receiver. All packets sent through the path. • Data received in the same order as it is sent. • TCP (Transmission Control Protocol) provides reliable communication. • Unreliable communication • Datagrams: Different packets are sent through different paths. • Data might be lost or out of sequence. • UDP (User datagram Protocol) provides unreliable communication. B. Prabhakaran