Chapter 4: Threads

1. Chapter 4: Threads

2. Objectives • To introduce the notion of a thread — a fundamental unit of CPU utilization that forms the basis of multithreaded computer systems • To discuss the APIs for the Pthreads, Win32, and Java thread libraries • To examine issues related to multithreaded programming

3. Threads Concept • So far we assumed a process was an executing program with a single thread of control. • Motivating example - A web server accepts client requests for Web pages, images, sound, and so forth (see example in next page) • A thread is a basic unit of CPU utilization (why CPU? What if it differed just in a specific device utilization?); it comprises: • Thread ID • Program counter • Register set • Stack • It shares with other threads belonging to the same process, its • Code section • Data section and • Other OS resources such as open files • If a process has multiple threads of control, it can perform more than one task at a time.

4. Multithreaded Server Architecture

5. Single and Multithreaded Processes Example – word processor: (a) displaying graphics, (b) responding to keystroke, (c) spell checker

6. Benefits • Responsiveness – allow a program to continue running even if part of it is blocked or performing a lengthy operation (e.g., web-browser loading an image) • Resource Sharing – threads share the memory and resources of the process (allows an application to have several threads of activity within the same address space) • Economy – it is more economical to create and context-switch threads (because they share resources of the process to which they belong) • In Solaris a process is thirty times slower to create and five times slower to context-switch • Scalability (utilizing multiprocessor architectures) – same process can make use of several CPUs by using threads

7. Multicore Programming • Multicore systems putting pressure on programmers, challenges include: • Dividing activities –find areas that can be divided into separate, concurrent tasks and thus can run in parallel on individual cores. • Balance – ensure tasks perform equal work of equal value • Data splitting – ensure that the data be divided to run on separate cores just like the application is divided into separate tasks. • Data dependency – ensure that the execution of the tasks is synchronized to accommodate data dependency. • Testing and debugging – more difficult than for single-threaded applications since there are many different execution paths when a program is running in parallel on multiple cores.

8. Concurrent Execution on a Single/Multi-core System

9. User and Kernel Threads • Kernel thread (sometimes called a Lightweight Process - LWP) is created and scheduled by the kernel: • More expensive to create than user threads • User thread is created by a threading library and scheduling is managed by the threading library itself (Which runs in user mode). • All user threads belong to process that created them. • Kernel is unaware of user-level threads so all thread • The major difference is when using multiprocessor systems: • User threads completely managed by the threading library can't be ran in parallel on the different CPUs (will run fine on uniprocessor systems) • Since kernel threads use the kernel scheduler, different kernel threads can run on different CPUs. • Programs can have user-space threads when threading with timers, signals, or other methods to interrupt their own execution, performing a sort of ad-hoc time-slicing.

10. User Threads • Thread management done by user-level threads library • Three primary thread libraries: • POSIX Pthreads • Win32 threads • Java threads

11. Kernel Threads • Supported by the Kernel • Examples • Windows XP/2000 • Solaris • Linux • Mac OS X

12. Multithreading Models • Many-to-One • One-to-One • Many-to-Many

13. Many-to-One • Many user-level threads mapped to single kernel thread • Thread management is done by the thread library in user space, so it is efficient. • Drawback - the entire process will block if a thread makes a blocking system call. • Examples: • Solaris Green Threads • GNU Portable Threads

14. One-to-One • Each user-level thread maps to kernel thread • More concurrency compared to many-to-one model since another thread will be able to run when a thread makes a blocking system call. • Allows multiple threads to run in parallel on multiprocessors. • Drawback: • Creating a user thread requires creating a corresponding kernel thread. • Performance will degrade and hence, most systems has a restriction on the number of threads allowed. • Examples • Windows NT/XP/2000 • Linux • Solaris 9 and later

15. Many-to-Many Model • Allows many user level threads to be mapped to many kernel threads • Allows the operating system to create a sufficient number of kernel threads • Solaris prior to version 9 • Windows NT/2000 with the ThreadFiber package

16. Two-level Model • Similar to M:M, except that it allows a user thread to be bound to kernel thread • Examples • IRIX • HP-UX • Tru64 UNIX • Solaris 8 and earlier

17. Thread Libraries • Thread library provides programmer with API for creating and managing threads • Two primary ways of implementing • Library entirely in user space • Kernel-level library supported by the OS

18. Pthreads • May be provided either as user-level or kernel-level • A POSIX standard (IEEE 1003.1c) API for thread creation and synchronization • API specifies behavior of the thread library, implementation is up to development of the library (i.e. the OS designers) • Common in UNIX operating systems (Solaris, Linux, Mac OS X)

19. Java Threads • Java threads are managed by the JVM • Typically implemented using the threads model provided by underlying OS • Java threads may be created by: • Extending Thread class (and override its run() method) • Implementing the Runnable interface • Calling start() • Allocates memory and initializes a new thread in the JVM • Calls the run() method, making the thread eligible to run on the JVM

20. Threading Issues • Semantics of fork() and exec() system calls • Thread cancellation of target thread • Asynchronous or deferred • Signal handling • Thread pools • Thread-specific data • Scheduler activations

21. Semantics of fork() and exec() • Does fork() duplicate only the calling thread or all threads? • If one thread in a program calls fork(), does the new process duplicate all threads, or is the new process single-threaded? Unix has both versions. • When exec() system call is executed, the program specified in its parameter will replace the entire process, including all the threads. Hence, duplicating only the calling thread is appropriate.

22. Thread Cancellation • Terminating a thread before it has finished • Example: multiple threads concurrently searching through a database and one thread returns the result • Stopping a webpage from loading any further • Two general approaches: • Asynchronous cancellation terminates the target thread immediately – may not free its resources • Deferred cancellation allows the target thread to periodically check if it should be cancelled – checking a flag

23. Signal Handling • Signals are used in UNIX systems to notify a process that a particular event has occurred • Example: illegal memory access, division by zero • A signal handler is used to process signals • Signal is generated by particular event • Signal is delivered to a process • Signal is handled • Options: • Deliver the signal to the thread to which the signal applies • Deliver the signal to every thread in the process • Deliver the signal to certain threads in the process • Assign a specific thread to receive all signals for the process

24. Thread Pools • Create a number of threads in a pool where they await work • Advantages: • Usually slightly faster to service a request with an existing thread than create a new thread • Allows the number of threads in the application(s) to be bound to the size of the pool • Unlimited threads could exhaust system resources, such as CPU time or memory

25. Scheduler Activations • Both M:M and Two-level models require communication to maintain the appropriate number of kernel threads allocated to the application • Such coordination allows the number of kernel threads to be dynamically adjusted to help ensure the best performance • Communication between the user-level thread library and the kernel is known as Scheduler activations. • Scheduler activations provide upcalls - a communication mechanism from the kernel to the thread library