CMP320 Operating Systems Lecture 09, 10 Operating System Concepts

1. CMP320Operating SystemsLecture 09, 10Operating System Concepts October 12, 14 2009 Arif Butt • Note: Some slides and/or pictures are adapted from Lecture slides / Books of • Dr Mansoor Sarwar. • Dr Kubiatowicz. • Dr P. Bhat. • Dr Hank Levy. • Dr Indranil Gupta. • Text Book - OS Concepts by Silberschatz, Galvin, and Gagne. • Ref Books • Operating Systems Design and Implementation by Tanenbaum. • Operating Systems by William Stalling. • Operating Systems by Colin Ritchie.

2. Today’s Agenda • Review of previous lecture • Introduction to Threads • Multi threading and Hyper threading • Merits and Demerits of Threads • User Level vs Kernel Level Threads • Threading Models • Many to One • One to One • Many to Many • Parallel Processor Architectures • Using POSIX pthread library calls to create multi-threaded programs CMP320 PUCIT Arif Butt

3. Issues with Processes • The fork() system call is expensive, copying the entire parent process as child process It may not always be required to copy entire parent process to child process as the child is going to call exec() system call immediately in most of the cases • IPC is required to pass information between a parent and its child processes CMP320 PUCIT Arif Butt

4. Processes and Threads • Processes have two characteristics: • Resource ownership - process includes a virtual address space to hold the process image • Scheduling/execution - follows an execution path that may be interleaved with other processes • These two characteristics are treated independently by the operating system • The unit of dispatching is referred to as a thread or lightweight process • The unit of resource ownership is referred to as a process or task CMP320 PUCIT Arif Butt

5. Thread Concept • A thread is a “lightweight” process which executes within the address space of a process • A thread is an execution context that is independently scheduled, but shares single addresses space with other threads of the same process CMP320 PUCIT Arif Butt

6. Single Threaded Process main() { … f1(…); … f2(…); … } f1(…) { … } f2(…) { … } Thread f1 f2 Process Terminated CMP320 PUCIT Arif Butt

7. Thread Concept (cont…) • Previous slide is an example of a process with a single thread • Suppose we want that f1 and f2 should be executed by separate threads, while main function is executed concurrently by another thread • Multi threading refers to the ability of an OS to support multiple threads of execution with in a single process.A single program made up of a number of different concurrent activities; sometimes called multitasking, as in Ada • Multithreading works similar to multiprogramming. The CPU switches rapidly back and forth among threads providing the illusion that threads are running in parallel CMP320 PUCIT Arif Butt

8. Multi Threaded Process main() { … thread(t1,f1); … thread(t2,f2); … } f1(…) { … } f2(…) { … } Process Address Space main t1 t2 PC PC PC CMP320 PUCIT Arif Butt

9. Single and Multithreaded Processes Environment (resource) execution (a) Three processes, each with one thread (b) One process with three threads Processes are used to group resources together; Threads are the entities scheduled for execution on the CPU. CMP320 PUCIT Arif Butt

10. Single and Multithreaded Processes CMP320 PUCIT Arif Butt

11. Single and Multithreaded Processes CMP320 PUCIT Arif Butt

12. Thread Concept CMP320 PUCIT Arif Butt

13. Single and Multithreaded Processes Each Thread has its own Stack • Each thread stack contains one frame for each procedure that has been called but not yet returned from • This frame contains the procedure’s local variables and their return address to use when the procedure call has finished • For example, if procedure X calls procedure Y and this one calls procedure Z, while Z is executing the frames for X, Y, Z will be on the stack • Each thread will generally call different procedures and thus has a different execution history CMP320 PUCIT Arif Butt

14. Threads vs Processes • Similarities • A thread can also be in one of many states like new, ready, running, blocked, terminated • Like processes only one thread is in running state (single CPU) • Like processes a thread can create a child thread • Differences • No “automatic” protection mechanism is in place for threads—they are meant to help each other • Every process has its own address space, while all threads within a process operate within the same address space CMP320 PUCIT Arif Butt

15. Benefits of Threads • Responsiveness.Multi-threaded servers (e.g., browsers) can allow interaction with user while a thread is formulating response to a previous user query (e.g., rendering a web page) • Resource sharing. Process resources (code, data, etc.) are shared by all threads. OS resources (PCB, PPFDT, etc.) are also shared by all threads • Economy. Take less time to create, schedule, and terminate a thread. Solaris 2: Thread Creation is thirty times faster than Process Creation and Thread Switching is five times faster than process switching • Performance. In multi-processor and multi-threaded architectures (e.g., Intel’s P-IV HT) each thread may run on a different processor in parallel CMP320 PUCIT Arif Butt

16. Disadvantages of Threads • Problems due to Shared memory • Race Problem • Mutual Exclusion needs to be implemented on shared memory to allow multiple threads to access data for write/update • Many library functions are not Thread Safe • For resource sharing, synchronization is needed between threads • Lack of Robustness • A severe error caused by one thread (e.g. segmentation fault) terminates the whole process CMP320 PUCIT Arif Butt

17. Problems with Threads • Consider the effects of the fork() system call, when the parent process has multiple threads, should the child also have those multiple threads? • If not, the process may not function properly, since all of them may be essential • If yes, what happens if a thread was blocked on reading the keyboard? Are two threads now blocked on the keyboard? When a line is typed, do both threads get a copy of it? Only the parent? Only the child? CMP320 PUCIT Arif Butt

18. Problems with Threads (cont…) • 2. Another class of problems is related to the fact that threads share many data structures: • What happens if one thread closes a file while another one is still reading from it? • What happens when one thread feels that there is too little memory and starts allocating more memory, soon afterwards another thread of the same process executes and do the same. Does the allocation happen once or twice? CMP320 PUCIT Arif Butt

19. Problems with Threads (cont…) 3. Error Reporting: In UNIX, after a system call ,the status of the call is put into a global variable, errno. What happens if a thread makes a system call, and before it is able to read errno, another thread makes a system call, wiping out the original value? 4. Stack Management: When stack overflow occurs, the kernel just provides more stack, automatically. When a process has multiple threads, it must also have multiple stacks. If the kernel is not aware of all these stacks, it cannot grow them automatically upon stack fault. In fact, it may not even realize that a memory fault is related to stack growth CMP320 PUCIT Arif Butt

20. Thread Usage: Word Processor Reformatting thread A thread that handles disk back ups; writes contents of RAM to disk periodically. A thread that interacts with user • What will happen when a user deletes one line from page#1 of a 800 page document and immediately after that to make a change on page#600, gives a command telling the word processor to go to that page. CMP320 PUCIT Arif Butt

21. Thread Usage: Web Server Request for pages comes in and the requested page is sent back to the client. • Dispatcher thread reads incoming requests from the NW. • After examining the request, it chooses an idle worker thread and hands it the request. It also wakes up the worker from blocked state to ready state. • Worker now checks to see if the request can be satisfied from the Web page cache, to which all threads have access. If not, it starts a read() operation to get the page from the disk and blocks until the disk operation completes. When the thread blocks on the disk operation, another thread is chosen to run, possibly the dispatcher, in order to acquire more work, or possibly another worker that is now ready to run. CMP320 PUCIT Arif Butt

22. Thread Usage: Web Server (cont…) • Rough outline of code for previous slide Dispatcher Thread Worker Thread CMP320 PUCIT Arif Butt

23. One Many # of addr spaces: # threads Per AS: One Many Classification • Real operating systems have either • One or many address spaces • One or many threads per address space MS/DOS, early Macintosh Traditional UNIX Embedded systems (Geoworks, VxWorks, JavaOS,etc) JavaOS, Pilot(PC) Mach, OS/2, Linux Win NT to XP, Solaris, HP-UX, OS X CMP320 PUCIT Arif Butt

24. Multithreading • The ability of an OS to support multiple, concurrent paths of execution within a single process. CMP320 PUCIT Arif Butt

25. Single Thread Approaches • MS-DOS supports a single user process and a single thread. • Some UNIX, support multiple user processes but only support one thread per process CMP320 PUCIT Arif Butt

26. Multithreading • Java run-time environment is a single process with multiple threads • Multiple processes and threads are found in Windows, Solaris, and many modern versions of UNIX CMP320 PUCIT Arif Butt

27. User Threads • Thread management is done by user-level threads libraries (eg, POSIX Pthread, Win32 threads, Java threads, Solaris2 Threads, Mach C Threads) and Kernel is not aware of the existence of threads • An application can be programmed to be multithreaded by using user level thread library, which contains code for: • Thread creation and termination • Thread scheduling • Saving and restoring thread context • Passing messages and data between threads • By default an application begins with a single thread, within a process managed by Kernel. Later the application may spawn a new thread within the same process by invoking spawn utility in the thread library CMP320 PUCIT Arif Butt

28. Implementing Threads in User Space • Thread library used. • Thread table maintained in user space. • All thread management is done in user space by library • Kernel knows nothing about threads. • E.g.: pthread library User-level Threads CMP320 PUCIT Arif Butt

29. Kernel Threads • Thread management done by kernel and Kernel is aware of threads • Kernel level threads are supported in almost all modern operating systems: • Windows NT/XP/2000 • Linux • Solaris • Tru64 UNIX • Mac OS X • There must be a relationship between user threads and kernel threads. There exist three common models of this interaction (1:1, M:1 and M:N) CMP320 PUCIT Arif Butt

30. Implementing Threads in the Kernel • OS knows about individual threads within each process. • Thread table maintained by kernel. • E.g.: Windows 2K/XP Kernel-level Threads CMP320 PUCIT Arif Butt

31. Thread Libraries • Thread libraries provides programmers with API for creating and managing threads • Two primary ways of implementing • Library entirely in user space • Kernel-level library supported by OS • Pthreadsmay be provided either as user-level or kernel-level. Pthread is a POSIX standard (IEEE 1003.1c) API for thread creation and synchronization • Java Threads managed by the JVM. Typically implemented using the threads model provided by underlying OS. Java threads may be created by: Extending Thread class, or by implementing the Runnable interface. If underlying OS is Windows, then implemented using Win32 API; if it is Linux, then Pthreads. CMP320 PUCIT Arif Butt

32. Multi Threading Models Many-to-One Model (User Level Threads) User–level Threads Kernel–level Thread CMP320 PUCIT Arif Butt

33. Multi Threading Models (…) Many-to-One Model (User lvl Threads) • Many user threads per kernel thread; i.e. Kernel sees just one thread • Advantages • Thread management is done in user space, so it is efficient. • Disadvantages • When a thread within a process makes a system call the entire process blocks. • Because only one thread can access kernel at a time, no parallelism on multiprocessors is possible. • Example:POSIX Pthreads, Solaris Green threads, GNU Portable Threads CMP320 PUCIT Arif Butt

34. Multi Threading Models (…) One-to-One Model (Kernel lvl threads) P1 P2 User–level Threads Kernel–level Threads CMP320 PUCIT Arif Butt

35. Multi Threading Models (…) • One-to-One Model (Kernel lvl threads) • One ULT per KLT, so Kernel sees all threads • Advantage: • When one thread within a process makes a blocking system call the other threads are not blocked • Multiple threads can run in parallel on multi-processors • Disadvantage • Creating a user thread requires creating the corresponding kernel thread. There is an overhead related with creating kernel thread which can be burden on the performance • Examples: Windows NT/XP/2000, Solaris 9 and later, OS/2, FreeBSD CMP320 PUCIT Arif Butt

36. Multi Threading Models (…) Many-to-Many M:N Model(Hybrid Scheme) P1 P2 P3 User–level Threads Kernel–level Threads CMP320 PUCIT Arif Butt

37. Multi Threading Models (…) Many-to-Many Model (Hybrid Scheme) • Multiple user threads are multiplexed over a smaller or equal number of kernel threads. This model is a compromise between 1:1 and M:1 • User threads in the M:N model normally float among kernel threads – that may run on whatever kernel thread is available when they become runnable • Advantage • Application can create as many user threads as needed • Kernel threads run in parallel on multiprocessors • If one thread makes a blocking system call, kernel can schedule another • Examples:Solaris prior to version 9, HP-UNIX, Windows NT/2000 with the ThreadFiber package CMP320 PUCIT Arif Butt

38. Pop up Threads (Servers) • Creation of a new thread when message arrives (a) before message arrives (b) after message arrives CMP320 PUCIT Arif Butt

39. User Level vs Kernel Level Threads Advantages of ULTw.r.t KLT • Thread switching is fast and CPU is not interrupted during thread switching (No mode switch). Kernel is scheduling the processes while user level library will be scheduling the thread • Fair Scheduling (at process level): If P1 has one Thread and P2 has 100 threads. Scheduling will be fair for both the processes since the kernel never know of the user level threads • User level threads can be used even if the underlying OS/platform doesn't support multithreading Disadvantages of ULTw.r.t KLT • When ever ULT makes a system call, Kernel takes it as the system call has been made by the process, so all the threads of the process are blocked. • In a pure ULT strategy, a multithreaded application cannot take the advantage of multiprocessors. CMP320 PUCIT Arif Butt

40. User Level vs Kernel Level Threads Advantages of KLT w.r.t ULT • When ever a KLT makes a system call, only that thread is blocked and the Kernel can schedule another thread of same process. So all the threads of the process are NOT blocked • In a pure KLT strategy, a multithreaded application can take the advantage of multiprocessors Disadvantages of KLT w.r.t ULT • Thread switching is slow as CPU is interrupted during thread switching (Mode switch occurs) • Un Fair Scheduling (at process level): If P1 has one Thread and P2 has 100 threads. Since the Kernel is aware of threads, so P2 is going to get more CPU slices CMP320 PUCIT Arif Butt

41. SMT / Hyperthreading • There are two forms of multithreading: Temporal and Simultaneous multithreading (HTT by MS). • In temporal multithreading, only one thread of instructions can execute in any given pipeline stage at a time. • In simultaneous multithreading, instructions from more than one thread can be executing in any given pipeline stage at a time. This is done without great changes to the basic processor architecture: the main additions needed are the ability to fetch instructions from multiple threads in a cycle, and a larger register file to hold data from multiple threads (Super scalar Architecture). • In short HTT/SMT is multithreading on a superscalar architecture. CMP320 PUCIT Arif Butt

42. SMT / Hyperthreading Simple superscalar pipeline. By fetching and dispatching two instructions at a time, a maximum of two instructions per cycle can be completed. CMP320 PUCIT Arif Butt

43. Parallelism (Traditional View) • Traditionally, the computer has been viewed as a sequential machine • A processor executes instructions one at a time in sequence • Each instruction is a sequence of operations • Two popular approaches to providing parallelism • Symmetric MultiProcessors (SMPs) • Clusters CMP320 PUCIT Arif Butt

44. Categories of Computer Systems • Single Instruction Single Data (SISD) stream • Single processor executes a single instruction stream to operate on data stored in a single memory • Single Instruction Multiple Data (SIMD) stream • Each instruction is executed on a different set of data by the different processors • Multiple Instruction Single Data (MISD) stream • A sequence of data is transmitted to a set of processors, each of execute a different instruction sequence (Never implemented) • Multiple Instruction Multiple Data (MIMD) • A set of processors simultaneously execute different instruction sequences on different data sets CMP320 PUCIT Arif Butt

45. Parallel Processor Architectures CMP320 PUCIT Arif Butt

46. Symmetric Multiprocessing • In a symmetric multiprocessor (SMP): • The kernel can execute on any processor allowing portions of kernel to execute in parallel • Typically each processor does self-scheduling from the pool of available processes or threads. • The kernel can be constructed as multiple processes or multiple threads, allowing portions of the kernel to execute in parallel. This complicates the OS. • It must ensure that two processors do not choose the same process and that processes are not somehow lost from the queue • Techniques must be employed to resolve and synchronize claims to resources CMP320 PUCIT Arif Butt

47. Typical SMP Organization CMP320 PUCIT Arif Butt

48. CMP320 PUCIT Arif Butt

49. Important POSIX System Calls Thread Management • Thread operations include thread creation, termination, synchronization (joins, blocking), scheduling, data management and process interaction. • A thread does not maintain a list of created threads, nor does it know the thread that created it. CMP320 PUCIT Arif Butt

50. System Call – pthread_create() CMP320 PUCIT Arif Butt