1 / 69

Multithreading Models and Multicore Programming Overview

Learn about threads and processes, multithreaded applications design, implicit threading approaches, Windows and Linux thread representation, and multicore programming challenges and benefits. Explore user and kernel threads, thread libraries, and Amdahl’s Law for performance analysis.

mcspadden
Download Presentation

Multithreading Models and Multicore Programming Overview

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter 4: Threads & Concurrency

  2. Chapter 4: Threads • Overview • Multicore Programming • Multithreading Models • Thread Libraries • Implicit Threading • Threading Issues • Operating System Examples

  3. Objectives • Identify the basic components of a thread, and contrast threads and processes • Describe the benefits and challenges of designng multithreaded applications • Illustrate different approaches to implicit threading including thread pools, fork-join, and Grand Central Dispatch • Describe how the Windows and Linux operating systems represent threads • Design multithreaded applications using the Pthreads, Java, and Windows threading APIs

  4. Motivation • Most modern applications are multithreaded • Threads run within application • Multiple tasks with the application can be implemented by separate threads • A thread to Update display • A thread to Fetch data • A thread to Spell checking • A thread to Answer a network request • Process creation is heavy-weight while thread creation is light-weight • Can simplify code, increase efficiency • Kernels are generally multithreaded

  5. Multithreaded Server Architecture

  6. Benefits of Threads • Responsiveness – may allow continued execution if part of a process is blocked, especially important for user interfaces • Resource Sharing – threads share resources of process, easier than shared memory or message passing • Economy – cheaper than process creation, thread switching lower overhead than context switching • Scalability – process can take advantage of multiprocessor architectures

  7. Single and Multithreaded Processes

  8. Multicore Programming • Multicore or multiprocessor systems are putting pressure on programmers, the challenges include: • Dividing activities • Balance load • Data splitting • Data dependency • Testing and debugging • Parallelism implies a system can perform more than one task simultaneously • Concurrency supports more than one task making progress • Single processor / core, scheduler providing concurrency • Pseudo parallelism • Multiple tasks are interleaved making it look parallel

  9. Concurrency vs. Parallelism • Concurrent execution on single-core system: • Parallelism on a multi-core system:

  10. Multicore Programming (Cont.) • Types of parallelism • Data parallelism – distributes subsets of the same data across multiple cores, same operation on each • Task parallelism – distributing threads across cores, each thread performing unique operation • As # of threads grows, so does architectural support for threading • CPUs have cores as well as hardware threads • IBM Power Systems S924 — 4U, 2× POWER9 SMT8, 8-12 cores per processor, up to 4 TB DDR4 RAM, PowerVM running AIX/IBM i/Linux. • INTEL 8th generation, Kaby Lake & Coffee Lake, 2-6 cores per processor, 1-2 SMT per core. Hyperthreading on the higher end. Native support for DDR4

  11. Data and Task Parallelism

  12. Amdahl’s Law • Identifies performance gains from adding additional cores to an application that has both serial and parallel components • S is serial portion; N processing cores • That is, if application is 75% parallel / 25% serial, moving from 1 to 2 cores results in speedup of 1.6 times • As N approaches infinity, speedup approaches 1 / S Serial portion of an application has a disproportionate effect on performance gained by adding additional cores. • But does the law take into account contemporary multicore systems?

  13. Amdhal’s Law • S = .25 and N = 2 1/(.25 + (1-.25/2)) 1/(.25 + (.75/2)) 1/(.25 + .375) 1/.625 • 1.6 speedup

  14. Amdahl’s Law

  15. User Threads and Kernel Threads • User threads - management done by user-level threads library • Three primary thread libraries: • POSIX Pthreads • Windows threads • Java threads • Kernel threads - Supported by the Kernel • Examples – virtually all general purpose operating systems, including: • Windows • Linux • Mac OS X • iOS • Android

  16. Multithreading Models • Multithreading Models refer to mapping user threads to kernel threads. • There are three common Multithreading Models for Kernel threads • Many-to-One • One-to-One • Many-to-Many

  17. Many-to-One • Many user-level threads mapped to single kernel thread • One thread blocking causes all to block • Multiple threads may not run in parallel on muticore system because only one may be in kernel at a time • Few systems currently use this model • Examples: • Solaris Green Threads • GNU Portable Threads

  18. One-to-One • Each user-level thread maps to kernel thread • Creating a user-level thread creates a kernel thread • More concurrency than many-to-one • Number of threads per process sometimes restricted due to overhead • Examples • Windows • Linux

  19. Many-to-Many Model • Allows many user level threads to be mapped to many kernel threads • Allows the operating system to create a sufficient number of kernel threads • Like thread pooling, associates user thread to kernel thread during scheduling • Windows with the ThreadFiber package, otherwise not common.

  20. Two-level Model • Similar to M:M, except that it allows a user thread to be bound to kernel thread • Examples • IRIX • HP-UX • Tru64 UNIX • Solaris 8 and earlier

  21. Thread Libraries • Thread libraryprovides programmer with API for creating and managing threads • Two primary ways of implementing • Library entirely in user space • Kernel-level library supported by the OS

  22. Pthreads • May be provided either as user-level or kernel-level threads • A POSIX standard (IEEE 1003.1c) API for thread creation and synchronization • Specification, not implementation • API specifies behavior of the thread library, implementation is up to development of the library • Common in UNIX operating systems (Solaris, Linux, Mac OS X) • Let’s see an example on the next slide • A Triangular number is the sum of the numbers from 1-n • A program using multiple threads creates a thread that is passed the n value and then waits for the thread to return.

  23. Pthreads Example

  24. Pthreads Example (Cont.)

  25. Pthreads API • pthread_t // thread data including id. • pthread_attr_t // Opaque struct. Uses functions to get/set attrs • pthread_attr_init(pthread_attr* attrs); • Returns the default thread attrs. • pthread_create(pthread_t* thread, const pthread_attr_t* attrs, void *(*start_routine) (void *), void *arg); • pthread_join(pthread_t thread, void** retval); • Blocks until the given thread returns. If retval is non-NULL, then the pthread_exit value is returned in retval • pthread_exit(void* retval); • Exits the thread, if the main thread calls pthread_exit rather than exit, the children can continue executing.

  26. Pthreads Code for Joining 10 Threads

  27. Windows Multithreaded C Program

  28. Windows Multithreaded C Program (Cont.)

  29. Java Threads • Java threads are managed by the JVM • Typically implemented using the threads model provided by underlying OS • Java threads may be created by: • Extending Thread class • Implementing the Runnable interface

  30. Java Threads Implementing Runnable interface: Creating a thread: Waiting on a thread:

  31. Java Multithreaded Program

  32. Java Multithreaded Program (Cont.)

  33. Java Executor Framework • Rather than explicitly creating threads, Java also allows thread creation around the Executor interface: • Should be execute(Callable command) as well. • The Executor is used as follows:

  34. Java Executor Framework

  35. Java Executor Framework (cont)

  36. Threads • What is shared between threads? • How does a thread context switch differ from a process context switch?

  37. Threads • What is shared between threads? • Text – code is shared • Heap – is shared • Global data - is shared • Stack is not shared • Thread local storage – behaves like static, global but not shared. • How does a thread context switch differ from a process context switch? • If the kernel schedules at the thread level, then all context switches are thread context switches, however… stack pointer, program counter and registers must be saved in both cases. • Threads share text, heap and global data which means that a thread context switch can leave all caches and virtual memory alone. • A process context switch must empty caches and reload virtual memory • A new thread/process must be selected to resume/run.

  38. Thread Models • What are the four thread models?

  39. Thread Models • What are the four thread models? • One to Many • Every process has exactly one thread, Threads are controlled at the user level by the thread library. Also known as co-routines. • One to One • Every user thread is represented by a kernel thread. Process Control Block is actually a thread control block (or task control block). • Many to Many • Every user thread runs on a kernel thread but kernel threads are assigned when scheduled. • Two level Model • Most threads follow the Many to Many model but high priority threads are assigned on a One to One model.

  40. Implicit Threading • Growing in popularity as numbers of threads increase, program correctness more difficult with explicit threads • Creation and management of threads done by compilers and run-time libraries rather than programmers • Five methods explored • Thread Pools • Fork-Join • OpenMP • Grand Central Dispatch • Intel Threading Building Blocks

  41. Thread Pools • Create a number of threads in a pool where they await work • Advantages: • Usually slightly faster to service a request with an existing thread than create a new thread • Allows the number of threads in the application(s) to be bound to the size of the pool • Separating task to be performed from mechanics of creating task allows different strategies for running task • i.e.Tasks could be scheduled to run periodically • Windows API supports thread pools:

  42. Java Thread Pools • Three factory methods for creating thread pools in Executors class:

  43. Java Thread Pools (cont)

  44. Fork-Join Parallelism • Multiple threads (tasks) are forked, and then joined.

  45. Fork-Join Parallelism • General algorithm for fork-join strategy:

  46. Fork-Join Parallelism

  47. Fork-Join Parallelism in Java

  48. Fork-Join Parallelism in Java

  49. Fork-Join Parallelism in Java • The ForkJoinTask is an abstract base class • RecursiveTask and RecursiveAction classes extend ForkJoinTask • RecursiveTask returns a result (via the return value from the compute() method) • RecursiveAction does not return a result

  50. OpenMP • Set of compiler directives and an API for C, C++, FORTRAN • Provides support for parallel programming in shared-memory environments • Identifies parallel regions – blocks of code that can run in parallel #pragma omp parallel Create as many threads as there are cores #pragma omp parallel for for(i=0;i<N;i++) { c[i] = a[i] + b[i]; } Run for loop in parallel

More Related