UNIX: Background and the Traditional Process and Kernel Abstractions

1. UNIX:Background and the Traditional Process and Kernel Abstractions Lecture 2 Tuesday 28/8/01 CS523S: Operating Systems

2. A Little History First: UNIX • Initial design by Ken Thompson, Dennis Ritchie and others at AT&T's Bell Telephone Laboratories (BTL) in 1969. • AT&T made the source available to Universities for research and educational use. • 1973 UNIX was rewritten in C resulting in Version 4. • The C language was also originally designed and developed for use on the UNIX system by Dennis Ritchie • C was evolved from 'B', developed by Thompson. CS523S: Operating Systems

3. UNIX History • AT&T sold UNIX to Novel; Novel passed the UNIX trademark to X/OPEN and sold source code to Santa Cruz Operation (SCO). • Plan 9 is AT&T's successor to UNIX • AT&T was unable to market UNIX as a product so they made the source code available to Universities for use in Research and Education. • Influential variant: Berkeley Software Distributions (BSD) distributed by the Computer Systems Research Group, University of California at Berkeley • Berkeley obtained UNIX from AT&T in December of 1974 CS523S: Operating Systems

4. UNIX History • UNIX ported to many different architectures • Microsoft and SCO collaborated to port UNIX to the Intel 8086 architecture: XENIX • AT&T purchased 20% of Sun: result is joint effort to develop SVR4 • In 1982 AT&T was broken up and was now able to market UNIX. They released System III in 1982 and System V the following year. • System V UNIX introduced virtual memory (different from BSD, called regions), IPC (shared memory, semaphores, message queues), remote file sharing, shared libraries and STREAMS. CS523S: Operating Systems

5. BSD UNIX • 2 BSD: text editor vi • 3 BSD: demand-paged virtual memory • 4.0BSD: performance improvements • 4.1BSD: job control, autoconfiguration • 4.2/4.3BSD: reliable signals, fast filesystem, improved networking (TCP/IP reference implementation), sophisticated IPC primitives • 4.4 BSD: stackable and extensible vnode interface, network filesystem, log-structured filesystem, other filesystems, POSIX support, and other enhancements. CS523S: Operating Systems

6. Commercialization • Interactive Systems first commercial (1977) • Microsoft and SCO release XENIX (1980) • 1982 Bill Joy left Berkeley and founded Sun Microsystems. • SunOS originall based on BSD 4.2 • SunOS 5 (Solaris 2.X) was a collaborative effort based on System V, release 4 (SVR4). • AIX from IBM • HP/UX from Hewlett Packard Corporation • ULTRIX from Digitial Equipment Corporation, followed by DEC OSF/1. DEC purchased by Compaq CS523S: Operating Systems

7. CMU and MACH • As UNIX grew, the kernel became large and complex - originally small and elegant. • Mid-1980 CMU researches began development of a new OS: • microkernel providing a small set of essential services. • Support UNIX API • Support uniprocessor and multiprocessors • supported distributed environments • collection of servers provide necessary functionality • New source so not encumbered by AT&T licenses • OSF1 and NextStep based on MACH CS523S: Operating Systems

8. Standards • Problem: Incompatible programming APIs and core service implementations across the different UNIX variants • Solution: Standard set of interfaces. Several standards exist that define the functions, syntax and semantics: • IEEE POSIX (Portable Operating System Interface for computing environments or portable OS for UNIX) specifications. • System V Interface Definition (SVID) from AT&T • X/Open Portability Guide (XPG) from the X/Open Consortium CS523S: Operating Systems

9. Standards • IEEE POSIX specifications. • 1003.1 (1990) system interfaces and headers • 1003.1b (1993) realtime extensions • 1003.1c (1995) threads extensions (pthreads) • 1003.1 (1996) includes 1003.1 (1990), 1003.1b (1993), 1003.1c (1995), 1003.1i (1995) • 1003.1g protocol independent interfaces • 1003.2 (1992) Shell and utilities • 1003.2a (1992) Interactive shell and utilities • SVID: SVID4 is for SVR4 • defines the System V programming interface. Must be conformant to call an OS SVR5. AT&T published the System V Verification Suite (SVVS) CS523S: Operating Systems

10. Standards • XPG - formed to develop the Common Applications Environment (CAE). • XPG3 - Superset of POSIX.1 1998 and utilities from SVID3 • XPG4 - Superset of POSIX.1 1990, POSIX.2 (1992), POSIX.2 (1992), and extension from XPG3 • XPG4v2 (SUS or single UNIX specification or spc 1170 or UNIX 95) - superset of XPG4 with widely used BSD interfaces • XNS4 (X/Open Network Services) - sockets and XTI extensions • SUSV2 (UNIX98)- superset of SUS, POSIX.1b (1993), POSIX.1c (1996) and ISO/IEC 9899 (C standard). Includes CDE specification • XNS5 - superset of LP64 - clean derivative of XNS4 CS523S: Operating Systems

11. char block Device drivers Traditional Kernel Architecture execution environment application trap libraries user System call interface kernel File subsystem IPC System Services Process control subsystem Buffercache scheduler memory hardware CS523S: Operating Systems

12. Basic Concepts and Terminology • Two privilege level: user and system • Each process has own "protected" address space • All process share same kernel space • Kernel space protected: requires special instruction sequence to change from user to kernel mode • Per process state (u_area) in kernel space • Per process kernel stack; kernel is re-entrant • System (Interrupt) versus process context CS523S: Operating Systems

13. Mode, Space and Context Privileged user kernel mode context process Application (user code) System calls Exceptions system space kernel X not allowed Interrupts System tasks UNIX uses only two privilege levels CS523S: Operating Systems

14. Overview • Process • compete for system resources • blocks if can not acquire a resource • Kernel: Implements the process model • manages resource - allocate resource to processes "optimally" • implements policy for resource allocation (sharing) • provide high-level abstract interface to resources • centralized management simplifies synchronization and error recovery CS523S: Operating Systems

15. The Process • Fundamental abstraction • Executes a sequence of instructions • Competes for resources • Address space - memory locations accessible to process • virtual address space: memory, disk, swap or remote devices • System provides features of a virtual machine • shared resources (I/O, CPU etc) • dedicated registers and memory CS523S: Operating Systems

16. The Process • Most created by fork or vfork • well defined hierarchy: one parent and zero or more child processes. The init process is at the root of this tree • different programs may be run during the life of a process by calling exec • processes typically terminate by calling exit CS523S: Operating Systems

17. User Running Kernel Running Runnable Stopped Zombie Initial Idle Stopped Asleep Asleep Process states fork system call, interrupt return fork swtch() exit swtch() wait sleep() continue stop wakeup stop stop continue wakeup CS523S: Operating Systems

18. Process Context • Address Space • text, data, stack, shared memory ... • Control information (u area, proc) • u area, proc structure, maps • kernel stack • Address translation maps • Credentials • user and group ids • Environment variables • variable=value • typically stored at bottom of stack CS523S: Operating Systems

19. Process Context (cont) • Hardware context • program counter • stack pointer • processor status word • memory management registers • FPU registers • Machine registers saved in u area's process control block (PCB) during context switch to another process. CS523S: Operating Systems

20. Kernel stack stack Process address space Data Text (shared) 0x00000000 Process Address Space (one approach) 0xffffffff Kernel address space 0x7fffffff CS523S: Operating Systems

21. proc struct Stack Stack Stack kernel stack/u area kernel stack/u area kernel stack/u area Data Data Data Text (shared) Text (shared) Text (shared) Big Picture: Another look kernel memory CS523S: Operating Systems

22. Process Control Information • U area. • Part of user space (above stack). • typically mapped to a fixed address. • contains info needed while running. • Can be swapped • Proc • contains info needed when not running. • Not swapped out. • traditionally fixed size table CS523S: Operating Systems

23. U Area PCB - HW context pointer to proc real/effective ids args, return values or errors to current syscall. Signal info file descriptor table controlling terminal vnode Proc structure process ID and group u area pointer process state queue pointers - scheduler, sleep, etc. Priority memory management info flags U area and Proc structures CS523S: Operating Systems

24. User Credentials • Every user is assigned a unique user id (uid) and group id (gid). • Superuser (root) has uid == 0 and gid == 0 • every process both a real and effective pair of Ids. • effective id => file creation and access, • real id => real owner of process. Used when sending signals. • Senders real or effective id must equal receivers real id CS523S: Operating Systems

25. The Kernel • program that runs directly on the hardware • loaded at boot time and initializes system, • creates some initial system processes. • remains in memory and manages the system • Resource manager/mediator • Time share (time-slice) the CPU, • coordinate access to peripherals, • manage virtual memory. • Synchronization primitives. • Well defined entry points: • syscalls, exceptions or interrupts. • Performs privileged operations. CS523S: Operating Systems

26. Entry into Kernel • Synchronous - kernel performs work on behalf of the process: • System call interface (UNIX API): central component of the UNIX API • Hardware exceptions - unusual action of process • Asynchronous - kernel performs tasks that are possibly unrelated to current process. • Hardware interrupts - devices assert hardware interrupt mechanism to notify kernel of events which require attention (I/O completion, status change, real-time clock etc) • System processes - scheduled by OS • swapper and pagedaemon. CS523S: Operating Systems

27. Trap or Interrupt Processing • Hardware switches to kernel mode, using the per/process kernel stack. • HW saves PC, Status word and possibly other state on kernel stack. • Assembly routine saves any other information necessary and dispatches event. • On completion a return from interrupt (RFI) instruction is performed. CS523S: Operating Systems

28. Interrupt handling • Asynchronous event such as disk I/O, network packet reception or clock “tick”. • Must be serviced in system context. • Must not block. • This does take CPU time away from the currently running process! CS523S: Operating Systems

29. Interrupt handling (cont) • Multiple interrupt priority levels (ipl), traditionally 0-7. • User processes and kernel operate at the lowest ipl level. • Higher level Interrupts can preempt lower priority ones. • The current ipl level may be set while executing critical code. CS523S: Operating Systems

30. Exception handling • Synchronous to the currently running process for example divide by zero or invalid memory access. • Must run the the current processes context. • An Interrupt can occur during the processing of an exception or trap. CS523S: Operating Systems

31. Software Interrupts • Interrupts typically have the highest priority in a system. • Software interrupts are assigned priorities above that of user processes but below that of interrupts. • Software interrupts are typically implemented in software. • Examples are callout queue processing and network packet processing. CS523S: Operating Systems

32. System Call Interface • Implemented as an assembly language stub. • Trap into kernel dispatch routine which may save additional state and invoke the high-level system call. • User privileges are verified and any data is copied into kernel (copyin()). • On return, copy out any user data (copyout()), check for an Asynchronous System Trap (signal, preemption, etc). CS523S: Operating Systems

33. Synchronization • Kernel is re-entrant. • only one active process at any given time (others are blocked). • Nonpreemptive. • Blocking operations • masking interrupts CS523S: Operating Systems

34. Blocking Operations • When resource is unavailable (possibly locked), process sets flag and calls sleep() • sleep places process blocked queue, sets state to asleep and calls swtch() • when resource released wakeup() is called • all sleep processes are woken and state set to runnable (placed on the runnable queues). • When running, process must verify resource is available. CS523S: Operating Systems

35. Process Scheduling • Preemptive round-robin scheduling • fixed time quantums • priority is adjusted by nice value and usage factor. • Processes in the kernel are assigned a kernel priority (sleep priority) which is higher than user priorities. • Kernel 0-49, user 50-127. CS523S: Operating Systems

36. Signals • Asynchronous events and exceptions • Signal generation using the kill() system call • Default operation or user specific handlers • sets a bit in the pending signals mask in the proc structure • All signals are handled before return to normal processing • System calls can be restarted CS523S: Operating Systems

37. Process creation • Fork • create a new process • copy of parents virtual memory • parent return value is child’s PID • child return value is 0 • exec • overwrite with new program CS523S: Operating Systems

38. Process Termination • exit() is called • closes open files • releases other resources • saves resource usage statistics and exit status in proc structure • wakeup parent • calls swtch • Process is in zombie state • parent collects, via wait(), exit status and usage CS523S: Operating Systems