CPS 310 Recitation: Unix Processes etc.

1. CPS 310 Recitation: Unix Processes etc. Jeff Chase Duke University http://www.cs.duke.edu/~chase/cps310

3. Operating Systems: The Classical View Each process has a private virtual address space and one or more threads. Programs run as independent processes. data data Protected system calls ...and upcalls (e.g., signals) Protected OS kernel mediates access to shared resources. Threads enter the kernel for OS services. The kernel code and data are protected from untrusted processes.

4. Key Concepts for Classical OS • kernel • The software component that controls the hardware directly, and implements the core privileged OS functions. • Modern hardware has features that allow the OS kernel to protect itself from untrusted user code. • thread • An executing instruction path and its CPU register state. • virtual address space • An execution context for thread(s) defining a name space for executing instructions to address data and code. • process • An execution of a program, consisting of a virtual address space, one or more threads, and some OS kernel state.

5. A process can have multiple threads int main(intargc, char *argv[]) { if (argc != 2) { fprintf(stderr, "usage: threads <loops>\n"); exit(1); } loops = atoi(argv[1]); pthread_t p1, p2; printf("Initial value : %d\n", counter); pthread_create(&p1, NULL, worker, NULL); pthread_create(&p2, NULL, worker, NULL); pthread_join(p1, NULL); pthread_join(p2, NULL); printf("Final value : %d\n", counter); return 0; } volatile int counter = 0; int loops; void *worker(void *arg) { inti; for (i = 0; i < loops; i++) { counter++; } pthread_exit(NULL); } data Much more on this later!

6. Example: Chrome browser [Google Chrome Comics]

7. Processes in the browser Chrome makes an interesting choice here. But why use processes? [Google Chrome Comics]

8. Problem: heap memory and fragmentation [Google Chrome Comics]

9. Solution: whack the whole process When a process exits, all of its virtual memory is reclaimed as one big slab. [Google Chrome Comics]

10. Processes for fault isolation [Google Chrome Comics]

11. [Google Chrome Comics]

12. Isolation We need protection domains and protected contexts (“sandboxes”), even on single-user systems like your smartphone. There are various dimensions of isolation for protected contexts (e.g., processes): • Faultisolation. One app or app instance (context or process) can fail independently of others. If it fails, the OS can kill the process and reclaim all of its memory, etc. • Performance isolation. The OS manages resources (“metal and glass”: computing power, memory, disk space, I/O throughput capacity, network capacity, etc.). Each instance needs the “right amount” of resources to run properly. The OS prevents apps from impacting the performance of other apps. E.g., the OS can prevent a program with an endless loop from monopolizing the processor or “taking over the machine”. (How?) • Security. An app may contain malware that tries to corrupt the system, steal data, or otherwise compromise the integrity of the system. The OS uses protected contexts and a reference monitor to check and authorize all accesses to data or objects outside of the context.

13. VM and files: the story so far Process (running program) Files on “disk” File system calls (e.g., open/read/write) globals Memory-mapped sections of program file text Thread Program heap Register context for main thread of process And its stack Anonymous segments (zero-fill) Root directory File tree stack Segments (regions) in Virtual Address Space

14. Upcall example: Unix signals • Signals are asynchronous notifications to a process that some event of interest to it has occurred. • A process may register signal handlersfor various events relating to the process. The signal handlers are procedures in user space. • To deliver a signal, the kernel redirects a user thread to execute a selected registered signal handler in user mode. • Unix signals take a default action if no handler is registered. • E.g., segmentation fault  die. Other actions: ignore, stop data data ...and upcalls (e.g., signals) Protected system calls

15. slowcat • Ctrl-z, fg, bg • Sleep(3) • Speed matching for pipes • File refcounts

16. Thread states and transitions We will presume that these transitions occur only in kernel mode. This is true in classical Unix and in systems with pure kernel-based threads. Before a thread can sleep, it must first enter the kernel via trap (syscall) or fault. Before a thread can yield, it must enter the kernel, or the core must take an interrupt to return control to the kernel. STOP wait running On entry to the running state, kernel code decides if/when/how to enter user mode, and sets up a suitable context E.g., for initial start, return from fault or syscall, or to deliver a signal. yield preempt sleep dispatch blocked ready wakeup

17. Thread states and transitions running “driving a car” yield/preempt STOP Scheduler governs these transitions. wait sleep “waiting for someplace to go” dispatch blocked ready wakeup “requesting a car” wait, STOP, read, write, listen, receive, etc. Sleep and wakeup are internal primitives. Wakeup adds a thread to the scheduler’s ready pool: a set of threads in the ready state.

18. Kernel Stacks and Trap/Fault Handling stack stack stack stack System calls and faults run in kernel mode on a kernel stack for the current thread. Threads execute user code on a user stack in user space (the process virtual address space). data Each thread has a second kernel stack in kernel space (VM accessible only in kernel mode). Kernel code running in P’s process context has access to P’s virtual memory. syscall dispatch table The syscall (trap) handler makes an indirect call through the system call dispatch table to the handler registered for the specific system call.

19. More analogies: threads and stacks stack stack • Threads drive their cores on paths across the stage. • Each thread chooses its own path. (Determined by its program.) • But they must leave some “bread crumbs” to find their way back on the return! • Where does a thread leave its crumbs? On the stack! • Call frames with local variables • Return addresses This means that each thread must have its own stack, so that their crumbs aren’t all mixed together.

20. Thread context • Each thread has a context (exactly one). • Context == values in the thread’s registers • Including a (protected) identifier naming its VAS. • And a pointer to thread’s stack in VAS/memory. • Each core has a context (at least one). • Context == a register set that can hold values. • The register set is baked into the hardware. • A core can change “drivers”: context switch. • Save running thread’s register values into memory. • Load new thread’s register values from memory. • Enables time slicing or time sharing of machine. CPU core R0 Rn x PC y SP registers

21. More analogies: context/switching 1 2 3 Page links and back button navigate a “stack” of pages in each tab. Each tab has its own stack. One tab is active at any given time. You create/destroy tabs as needed. You switch between tabs at your whim. Similarly, each thread has a separate stack. The OS switches between threads at its whim. One thread is active per CPU core at any given time. time 

22. Kernel/user transitions for fork/exec/exit The kernel may start and kill threads, and/or arbitrarily change the user virtual memory and/or thread context. It does it all the time. fork exec wait fork syscall fork return wait* call wait* return parent time time child EXIT fork entry to user mode exit syscall exec* syscall entry to user mode in main

23. About the previous slide • A trap is a system call, e.g., fork/exec/exit/wait/ or open/close/read/write or pipe/dup2. • A program is an executable file that may be launched in a process, e.g., with an exec* system call.  When a program is running in a process that program controls the process.  More precisely, the program controls the behavior of a thread in the process while that thread is running in user mode.  • When I say that "a program invokes a system call" or "a process does a trap" I mean that a thread is running in a user program in a process, and that thread executes a trap instruction in the program, for the purpose of entering the kernel to perform a system call. In the example: • Exec* system call is invoked by the parent program running in the child process. • Exec* system call "returns" into the program whose name was the first argument to exec*.  That is the program I call the "child program": it  is now running in the child process, having replaced the parent programin the child process.  After exec*, the child program begins executing in its main().  (Be sure you understand how that happened.)

24. What does this code do? int main(intargc, char *argv[]) { printf("about to run program %s.\n", argv[0]); execve(argv[0], argv, 0); perror("exec failed"); }

25. Two threads stack stack Virtual memory “on deck” and ready to run program x code library running thread data R0 CPU (core) Rn y x PC y SP registers Register values saved in memory

26. Thread context switch stack stack switch out switch in Virtual memory program x code library data R0 1. save registers CPU (core) Rn y x PC y SP registers 2. load registers Running code can suspend the current thread just by saving its register values in memory. Load them back to resume it at any time.

27. What causes a context switch? There are three possible causes: • Preempt (yield). The thread has had full use of the core for long enough. It has more to do, but it’s time to let some other thread “drive the core”. • E.g., timer interrupt, quantum expired  OS forces yield • Thread enters Ready state, goes into pool of runnable threads. • Exit. Thread is finished: “park the core” and die. • Block/sleep/wait. The thread cannot make forward progress until some specific occurrence takes place. • Thread enters Blocked state, and just lies there until the event occurs. (Think “stop sign” or “red light”.) STOP wait

28. Switching out • What causes a core to switch out of the current thread? • Fault+sleep or fault+kill • Trap+sleep or trap+exit • Timer interrupt: quantum expired • Higher-priority thread becomes ready • …? switch out switch in run thread Note: the thread switch-out cases are sleep, forced-yield, and exit, all of which occur in kernel mode following a trap, fault, or interrupt. But a trap, fault, or interrupt does not necessarily cause a thread switch!

29. What cores do Idle loop scheduler getNextToRun() idle pause nothing? get thread put thread sleep? exit? ready queue (runqueue) timer quantum expired? got thread switch out switch in run thread

30. A simple program: parallel … int main(…arg N…) { for 1 to N dofork(); for 1 to N wait(…); } void child() { BUSYWORK {x = v;} exit(0); } … Parallel creates N child processes and waits for them all to complete. Each child performs a computation that takes, oh, 10-15 seconds, storing values repeatedly to a global variable, then it exits. How does N affect completion time? chase$ cc –o parallel parallel.c chase$ ./parallel ??? chase$

31. A simple program: parallel Three different machines Completion time (ms) N (# of children)

32. Parallel: some questions • Which machine is fastest? • How does the total work grow as a function of N? • Does completion time scale with the total work? Why? • Why are the lines flatter for low values of N? • How many cores do these machines have? • Why is the timing roughly linear, even for “odd” N? • Why do the lines have different slopes? • Why would the completion time ever drop with higher N? • Why is one of the lines smoother than the other two? • Can we filter out the noise?

33. The kernel syscall trap/return fault/return system call layer: files, processes, IPC, thread syscalls fault entry: VM page faults, signals, etc. thread/CPU/core management: sleep and ready queues memory management: block/page cache sleep queue ready queue I/O completions timer ticks interrupt/return

34. The kernel syscall trap/return fault/return system call layer: files, processes, IPC, thread syscalls fault entry: VM page faults, signals, etc. thread/CPU/core management: sleep and ready queues memory management: block/page cache policy policy sleep queue ready queue I/O completions timer ticks interrupt/return

35. Unix, looking backward: UI+IPC • Conceived around keystrokes and byte streams • User-visible environment is centered on a text-based command shell. • Limited view of how programs interact • files: byte streams in a shared name space • pipes: byte streams between pairs of sibling processes

36. X Windows (1985) • Big change: GUI. • Windows • Window server • App events • Widget toolkit

37. Files: hierarchical name space root directory applications etc. mount point external media volume or network storage user home directory

39. catserver ... structsockaddr_insocket_addr; sock = socket(PF_INET, SOCK_STREAM, 0); memset(&socket_addr, 0, sizeofsocket_addr); socket_addr.sin_family = PF_INET; socket_addr.sin_port = htons(port); socket_addr.sin_addr.s_addr = htonl(INADDR_ANY); if (bind(sock, (structsockaddr *) &socket_addr, sizeofsocket_addr) < 0) { perror("bind failed"); exit(1); } listen(sock, 10); while (1) { intacceptsock = accept(sock, NULL, NULL); forkme(acceptsock, prog, argv); close(acceptsock); } }

40. void forkme(int s, char* prog, char* argv[]) { int status; intrc; rc = fork(); if (rc < 0) { perror("fork failed: "); exit(1); } else if (rc == 0) { printf("I am a child: %d.\n", getpid()); dup2(s, 0); dup2(s, 1); dup2(s, 2); execve(prog, argv, 0); /* NOTREACHED */ perror("execve failed"); exit(1); } else { /* Comment out this waitpid to let the server handle many concurrent requests. */ if (waitpid(rc, &status, 0) == -1) perror("waitpid failed"); else printf("Child %d exited with status %d.\n", rc, WEXITSTATUS(status)); } }

41. /* * Save context of the calling thread (old), restore registers of * the next thread to run (new), and return in context of new. */ switch/MIPS (old, new) { old->stackTop = SP; save RA in old->MachineState[PC]; save callee registers in old->MachineState restore callee registers from new->MachineState RA = new->MachineState[PC]; SP = new->stackTop; return (to RA) } This example (from the old MIPS ISA) illustrates how context switch saves/restores the user register context for a thread, efficiently and without assigning a value directly into the PC.

42. Example: Switch() Save current stack pointer and caller’s return address in old thread object. switch/MIPS (old, new) { old->stackTop = SP; save RA in old->MachineState[PC]; save callee registers in old->MachineState restore callee registers from new->MachineState RA = new->MachineState[PC]; SP = new->stackTop; return (to RA) } Caller-saved registers (if needed) are already saved on its stack, and restored automatically on return. Switch off of old stack and over to new stack. Return to procedure that called switch in new thread. RA is the return address register. It contains the address that a procedure return instruction branches to.

43. What to know about context switch • The Switch/MIPS example is an illustration for those of you who are interested. It is not required to study it. But you should understand how a thread system would use it (refer to state transition diagram): • Switch() is a procedure that returns immediately, but it returns onto the stack of new thread, and not in the old thread that called it. • Switch() is called from internal routines to sleep or yield (or exit). • Therefore, every thread in the blocked or ready state has a frame for Switch() on top of its stack: it was the last frame pushed on the stack before the thread switched out. (Need per-thread stacks to block.) • The thread create primitive seeds a Switch() frame manually on the stack of the new thread, since it is too young to have switched before. • When a thread switches into the running state, it always returns immediately from Switch() back to the internal sleep or yield routine, and from there back on its way to wherever it goes next.

44. Messing with the context #include <ucontext.h> int count = 0; ucontext_tcontext; int main() { inti = 0; getcontext(&context); count += 1; i += 1; sleep(2); printf(”…", count, i); setcontext(&context); } ucontext Standard C library routines to: Savecurrent register context to a block of memory (getcontextfrom core) Load/restore current register context from a block of memory (setcontext) Also: makecontext, swapcontext Details of the saved context (ucontext_t structure) are machine-dependent.

45. Messing with the context (2) #include <ucontext.h> int count = 0; ucontext_tcontext; int main() { inti = 0; getcontext(&context); count += 1; i += 1; sleep(1); printf(”…", count, i); setcontext(&context); } Save CPU core context to memory Loading the saved context transfers control to this block of code. (Why?) What about the stack? Load core context from memory

46. Messing with the context (3) #include <ucontext.h> int count = 0; ucontext_tcontext; int main() { inti = 0; getcontext(&context); count += 1; i += 1; sleep(1); printf(”…", count, i); setcontext(&context); } chase$ cc -o context0 context0.c < warnings: ucontext deprecated on MacOS > chase$ ./context0 1 1 2 2 3 3 4 4 5 5 6 6 7 7 …

47. Reading behind the C On MacOS: chase$ man otool chase$ otool –vt context0 … count += 1; i+= 1; On this machine, with this cc: Static global _count is addressed relative to the location of the code itself, as given by the PC register [%rip is instruction pointer register] Local variable i is addressed as an offset from stack frame. [%rbp is stack frame base pointer] Disassembled code: movl 0x0000017a(%rip),%ecx addl $0x00000001,%ecx movl %ecx,0x0000016e(%rip) movl 0xfc(%rbp),%ecx addl $0x00000001,%ecx movl %ecx,0xfc(%rbp) %rip and%rbpare set “right”, then these references “work”.

48. Messing with the context (4) #include <ucontext.h> int count = 0; ucontext_tcontext; int main() { inti = 0; getcontext(&context); count += 1; i += 1; sleep(1); printf(”…", count, i); setcontext(&context); } chase$ cc –O2 -o context0 context0.c < warnings: ucontext deprecated on MacOS > chase$ ./context0 1 1 2 1 3 1 4 1 5 1 6 1 7 1 … What happened?

49. The point of ucontext • The system can use ucontext routines to: • “Freeze” at a point in time of the execution • Restart execution from a frozen moment in time • Execution continues where it left off…if the memory state is right. • The system can implement multiple independent threads of execution within the same address space. • Create a context for a new thread with makecontext. • Modify saved contexts at will. • Context switch with swapcontext: transfer a core from one thread to another (“change drivers”) • Much more to this picture: need per-thread stacks, kernel support, suspend/sleep, controlled ordering, etc.

50. Messing with the context (5) #include <ucontext.h> int count = 0; ucontext_tcontext; int main() { inti = 0; getcontext(&context); count += 1; i += 1; sleep(1); printf(”…", count, i); setcontext(&context); } What does this do?