CSE 5343/7343 Fall 2006

1. CSE 5343/7343Fall 2006 Case Studies UNIX UNIX Case Study

2. UNIX Case Study Outline • History • Design Philosophy • Process Management • Processes • PCB/Process Table/U Area • Process States • Process Scheduling/Priorities • IPC • Memory Management • File Systems UNIX Case Study

3. History ([2],[3]) • 1965 – Bell and GE joined project MAC at MIT to develop Multics. Goal: multicomputing and data sharing for a large group of users. • Ken Thompson (Bell) developed a game called “Space Travel” and found an unused PDP-7. • 1969 • Thompson and Ken Ritchie implemented an earlier designed file system on PDP-7. • This grew into UNIX. • File system design and idea of command interpreter (shell) as a user process came from Multics • Fork idea came from Berkeley’s GENIE OS. UNIX Case Study

4. History (cont’d) • 1971 – UNIX used in first real project: text processing for BELL labs patent department. • 1971 – UNIX required 16K bytes for system, 8K bytes for user programs, 512K bytes of disk, and limit of 64K disk bytes per file. • Thompson set out to write a new Fortran compiler but developed a new programming language: B. B was based on BCPL (a tool for compiler writing and systems programming). B was interpretive. He later improved on B and called it C. • 1973 – UNIX rewrittten in C (unheard of at the time). AT&T offered UNIX free to universities. • 1974 – Ritchie and Thompson paper describingUNIX in CACM. UNIX Case Study

5. History (cont’d) • 1977 - Bell combined several versions into UNIX System II and marketed. History (cont’d) • 1978 – After distribution of Ver7 in 1978, the Unix Support Group (USG) at AT&T took over responsibilities from the research for distribution within AT&T. • 1982 – First external distribution from USG – System III. • UC Berkeley developed their version of UNIX (Berkeley software Distributions). • BSD introduced vi in 2BSD, demand-pages virtual memory in 3BSD, TCP/IP networking protocol in 4.2BSD. • Less than 3% of BSD written in assembly. • 1991 – Finnish student Linus Torvalds wrote Linux for 80386, 32 bit processor UNIX Case Study

6. Design Philosophy ([2],[3]) • Smplicity • C • Time-sharing • Simple user interface (modular and may be replaced) • Device independence (treat files and devices in same manner) • Aimed for programming environment • Flexibility UNIX Case Study

7. Design (cont’d) • Programs such as shell and vi interact with kernel using well defined system call procedures. • Cc built on top of c preprocessor, two-pass compiler. UNIX Case Study

8. UNIX Process Management UNIX Case Study

9. Processes • Process is program in execution • Consists of machine instructions (text), data, and stack regions. • Separate stack for user and kernel mode. • PID (Process ID) • Processes are either user processes, daemon processes, or kernel processes. • Daemons are not associated with user, but do system wide functions. Init may create daemons that exist throughout the life of the system or as needed. UNIX Case Study

10. Solaris Threads ([3]) • Kernel and user level threads • Only kernel level threads are scheduled • Implements Pthread API • LWP between kernel and user level threads • Each LWP associated with a kernel thread. • User processes have at least one LWP. • User threads may be bound or unbound to a LWP. • May have kernel thread without LWP. • Pool of LWPs for a process UNIX Case Study

11. Solaris Threads (cont’d) • Kernel thread • Copy of kernel registers • Pointer to LWP • Priority • Scheduling information • Stack • LWP • Registers • Memory • Accounting information • User level thread • Thread ID • Registers • Stack pointer/stack • Priority • Process • Process ID • Memory map • Open files • Priority • LWPs UNIX Case Study

12. PCB ([1],[2],[4]) • Process Table: • State • UID for owner • Memory status (swapped or in memory) • Parent PID • Child PID • Event descriptor if suspended • Entry in kernel process table that points to process region table. These in turn point to entries in the region table and to the regions for that process. This allows independent processes to share regions. UNIX Case Study

13. U (User) Area ([2]) • Information needed when process is executing. • Process table slot • Information about current system call • File descriptors for open files • Current directory • Current root • Login terminal • Kernel has direct access to U area of currently executing process. UNIX Case Study

14. Process States ([1],[2]) • No context switch is needed to go from state 2 to state 1. • States 3 and 7 are really the same. • I/O request puts process to sleep. • Zombie state – Child has finished execution, but parent wants to get information about it from the PCB. UNIX Case Study

15. Process Hierarchy ([1]) • Initial boot process (0) forks a child (process 1) and process 0 becomes the swapper. Process 1 is init process and is ancestor of all other processes. • Parent/Child/Sibling relationship indicated by pointers in process table. • Execve usually called to replace memory portion of new process. • Exit – process termination • Wait – Parent waits for child to exit. Reclaims process resources. • Process group ID (GID) in process table • Execute: ps -alx UNIX Case Study

16. Process Scheduling([2],[3],[5]) • Typical time slice values: • 4.3BSD: 1/10 second • System V: 50-100 times per second • Round robin multilevel feedback queue • At end of context switch, kernel executes algorithm to schedule a process. • Highest priority process which is ready is scheduled. On tie, picks the one which has waited the longest. UNIX Case Study

17. Process Scheduling (cont’d) • Higher number – Lower Priority • User and kernel priorities; user below kernel. • User mode priority is function of recent CPU usage. Lower priority if recently used CPU. • Priority assigned based on process group UNIX Case Study

18. Process Scheduling (cont’d) • Kernel to user mode: change to user mode and calculate based on kernel resources just used. • Clock Handler adjusts all user mode process priorities at 1 second (System V) intervals and causes kernel to reschedule. • Decay function adjusts recent CPU usage values at this time: decay(CPU) = CPU/2 • Priority is recalculated at these 1 second intervals plus when a process is in the preempted but ready to run state: priority = CPU/2 + base level priority • Base level priority is the threshold between user and kernel mode. • Processes move up in queue but can not change to kernel mode. UNIX Case Study

19. IPC ([2],[3]) • Pipes – Only used from descendents of process that created the pipe. • Named Pipes- Used by unrelated processes. Has a directory entry and is accessed as a file. • Signals • Inform process of occurrence of asynchronous events. • Handling of signal by user process. Address of this routine is located in uarea next to signal number. • Socket • Introduced by 4.2BSD • Transient object; Exists only as long as some process holds a descriptor referring to it. • Created by socket system call UNIX Case Study

20. IPC (cont’d) • Messages (System V) • On sending a message, a new entry is added to the associated linked list and the message is copied form the uarea. Message headers arranged in FIFO order. • Kernel awakens processes waiting for a message from this queue. • On receiving a message, process indicates what should be done if no message on the queue. UNIX Case Study

21. IPC (cont’d) • Shared Memory (System V) • System call creates a new region of shared memory or returns a current one. • Reading and writing is done as normal – no system calls. • Shared memory remains in tact even if no processes include it in their virtual address space. UNIX Case Study

22. IPC (cont’d) • Semaphores (System V) • System calls create and use semaphore. • Semaphore array where each entry is the count value. • If process is put to sleep, it sleeps at an interruptable priority and wakes up on receipt of a signal. • Handled similar to messages with id being the entry in the semaphore table. UNIX Case Study

23. UNIX Memory Management UNIX Case Study

24. Memory Management ([1],[2]) • Depends on platform and version • Early versions • Swapping • Transferred entire processes • 3BSD – First demand-paged VM UNIX Case Study

25. Swapping ([2]) • Kernel swaps out a process if more memory space needed: • Fork system call must allocate space for child • Size of process increases • Kernel wants to free memory for processes previously swapped out • Picking target process: • Zombie are not swapped (they take up no memory) • Sleeping swapped out before ready to run • Depends on Priority and time in memory • If no sleeping process then ready to run based on nice value and time in memory UNIX Case Study

26. Swapping (cont’d) • Nice Value used to change to scheduling priority • Process 0 – Swapper • Highest scheduling priority • Problem – thrashing: UNIX Case Study

27. Demand Paging – System V [2] • Approximate working set • Additional Page Table entries Valid • Reference • Modify • Copy on Write – New copy must be created if updated • Age – Time in working set • Disk Block Descriptor • Location on disk (VM) • Demand fill – Immediately overwrite contents during exec • Demand zero – Clear contents UNIX Case Study

28. Address Translation (4.3BSD Vax)[1] • Each region of address space mapped to a separate pag etable • Separate User and system area page tables • Two-level mapping, first level page tables reside in virtual memory • Page is 512 bytes • Virtual address • First two bits indicate regions • 10 – system • 01 – P0 - stack (user stack, kernel stack) • 00 – P1 - Heap, data, text • User translation requires two levels of translation UNIX Case Study

29. Page Replacements – Clock (4.3) [1] UNIX Case Study

30. Memory Address Space Frame Table Address Space Free Ref PID Backing Store Frame Update Disk Valid Clock Algorithm Example Process 1 PT 0 1 2 3 4 Frame Update Disk Valid Process 2 PT UNIX Case Study

31. Memory Frame Table 1 1 1 1 1 Free Ref PID Backing Store Start State 0 1 2 3 4 UNIX Case Study

32. Memory Address Space Frame Table 1 1 1 1 1 Free Ref PID Backing Store Create Process 1 / Load into VM Process 1 PT ^ ^ ^ ^ 0 0 0 0 0 1 2 3 4 A B C D Frame Update Disk Valid UNIX Case Study

33. Memory Address Space Backing Store Load 1st Two Pages of P1 into Memory Process 1 PT 1 3 0 0 ^ ^ ^ ^ 1 1 0 0 A B 0 1 2 3 4 A B C D Frame Table 1 0 1 0 1 1 1 Frame Update Disk Valid Ref PID Free UNIX Case Study

34. Memory Address Space Backing Store P1 Begins Executing using Pages 0&1 Process 1 PT 1 3 0 0 ^ ^ ^ ^ 1 1 0 0 A B 0 1 2 3 4 A B C D Frame Table 1 0 1 0 1 1 1 1 1 Frame Update Disk Valid Ref PID Free UNIX Case Study

35. Memory Address Space Address Space Backing Store Frame Update Disk Valid Create Process 2 / Load into VM Process 1 PT 1 3 0 0 ^ ^ ^ ^ 1 1 0 0 A B 0 1 2 3 4 A B C D Frame Table 1 0 1 0 1 1 1 1 1 Frame Update Disk Valid Process 2 PT Ref PID Free D E F ^ ^ ^ 0 0 0 UNIX Case Study

36. Memory Address Space Address Space Backing Store Frame Update Disk Valid Load 1st Two Pages of P2 into Memory Process 1 PT 1 3 0 0 ^ ^ ^ ^ 1 1 0 0 E A D B 0 1 2 3 4 A B C D Frame Table 0 0 0 0 1 0 1 0 1 2 1 2 1 Frame Update Disk Valid Process 2 PT Ref PID Free D E F 0 0 ^ ^ ^ 2 0 1 1 0 UNIX Case Study

37. Memory Address Space Address Space Backing Store Frame Update Disk Valid P2 Begins Executing using Pages 0&1 – Update Page 1 Process 1 PT 1 3 0 0 ^ ^ ^ ^ 1 1 0 0 E A D B 0 1 2 3 4 A B C D Frame Table 0 0 0 0 1 1 1 1 1 2 1 2 1 Frame Update Disk Valid Process 2 PT Ref PID Free D E F 0 1 ^ ^ ^ 2 0 1 1 0 UNIX Case Study

38. Memory Address Space Address Space Backing Store Frame Update Disk Valid P2 Page Fault for Page 2 – P1 Executes During Paging I/O Process 1 PT 1 3 0 0 ^ ^ ^ ^ 1 1 0 0 E A D B F 0 1 2 3 4 A B C D Frame Table 0 0 0 0 0 1 1 1 1 0 2 1 2 1 2 Frame Update Disk Valid Process 2 PT Ref PID Free D E F 0 1 0 ^ ^ ^ 2 0 4 1 1 1 UNIX Case Study

39. Memory Address Space Address Space Backing Store Frame Update Disk Valid I/O Completion Interrupt – P2 Begins Execution - Updates Page 2 Process 1 PT 1 3 0 0 ^ ^ ^ ^ 1 1 0 0 E A D B F 0 1 2 3 4 A B C D Frame Table 0 0 0 0 0 1 1 1 1 1 2 1 2 1 2 Frame Update Disk Valid Process 2 PT Ref PID Free D E F 0 1 1 ^ ^ ^ 2 0 4 1 1 1 UNIX Case Study

40. PT Memory 1 3 0 0 ^ ^ ^ ^ 1 1 0 0 Address Space A B C D Frame Table 0 0 0 0 0 0 1 1 1 1 2 1 2 1 2 Frame Update Disk Valid Process 2 Address Space PT Ref PID Free D E F 0 1 2 ^ ^ ^ 2 0 4 1 1 1 Backing Store 0 0 0 1 1 0 0 1 1 1 0 0 0 0 0 0 0 0 0 1 Frame Update Disk Valid Timer Interrupt – P1 Begins Execution – Page Fault for Page 2 – No Free Frames – Initiate Clock Algorithm for Page Replacement Process 1 E A D B F 0 1 2 3 4 UNIX Case Study

41. Memory Address Space Address Space Backing Store Frame Update Disk Valid Replace Frame 0 – Must swap out to disk as updated Process 1 PT 1 3 0 0 0 0 ^ ^ ^ ^ 1 1 1 0 C A D B F 0 1 2 3 4 A B C D Frame Table 0 0 0 0 0 0 0 0 0 0 1 1 2 1 2 Frame Update Disk Valid Process 2 PT Ref PID Free D E F 0 1 2 ^ ^ ^ 2 0 4 1 0 1 UNIX Case Study

42. References • Samuel J. Leffler, Marshall Kirk McKusick, Michael J. Karels, and John S. Quarterman, The Design and Implementation of the 4.3 BSD UNIX Operating System, Addison-Wesley, 1989. • Maurice J. Bach, The Design of the UNIX Operating System, Prentice Hall, 1990. • Abraham Silberschatz, Peter Baer Galvin, and Greg Gagne, Operating System Concepts Sixth Edition, John Wiley & Sons, 2002. • Milan Milenkovic, Operating Systems Concepts and Design Second Edition, McGraw Hill, 1992. • Andrew S. Tanenbaum, Modern Operating Systems, Prentice Hall, 1992. UNIX Case Study