Multithreading

1. Multithreading 05.05.2014

2. Overview • Definitions • Process • Thread • Multithread • Synchronization • Multithread Programming • Synchronization in Threads

3. Process • What is a Process? • A process can be defined as a program in execution • Each process has a separate address space • program code and its current activity • More than one process can be active at a given time • The OS facilitates sharing of computer resources by many processes concurrently in a secure way • To give the illusion that many processes are executing concurrently, the CPU switches between processes (contextswitching)

4. Process • What is a process from a programmers point of view? • An envelope • Contains the variables, resources and Threads of an application • Every process has at least one path of execution i.e. Main thread which is called upon start

5. Simple Execution Start Executing from first instruction in main thread Locate Main Thread Start Process Exit and Return Continue Till end

6. Is This enough? • What happened in all the applications you implemented till now • One instruction at a time, executing instructions one by one • How about concurrent operations? • Execution while asking for input? • Producing and consuming data? • …

7. Threads and Multiple-Threads • A thread is a “path of execution” in a process • Single sequential flow of control within a program • Thread is similar to a process, however • Multiple threads can share the same address space

8. Threads and Multiple-Threads • Every process has at least one thread : The main Thread • In your applications this would be the “main” function • Multitasking allows single processor to run several concurrent threads • Starting from this main thread you can spawn “child” threads • Child thread : The spawned thread • Parent thread: the thread that spawned

9. Why Threads? • To maintain responsiveness of an application during a long running task. • To enable cancellation of separable tasks. • Some problems are intrinsically parallel. • To monitor status of some resource : DB. • Some APIs and systems demand it: GUI Libraries. • To take advantage of multiple processors.

10. Example use of threads compute thread I/O thread compute I/O request I/O I/O complete I/O result Needed I/O result Needed compute (a) Sequential process (b) Multithreaded process

11. Threads • Remember That: • A thread is not a new Process • All threads of a process share the virtual address space, global variables, and operating system resources of the process

12. Advantages of Multithreading • Speed • Communication between threads is very fast • I/O throughput • A blocked I/O request only puts the calling thread to sleep • The other threads go about their business

13. Threads and CPU CPU Process 1 Process 2 Process 3 Process 4 Process 5 Process 6 Thread 1 Thread 1 Thread 2 Thread 2 Thread 3

14. Multithreads • These threads run concurrently using “scheduling” mechanisms of the operating system

15. Scheduling • Scheduling is a key concept in computer multitasking, multiprocessing operating system and real-time operating system designs • Scheduling refers to the way processes are assigned to run on the available CPUs, since there are typically many more processes running than there are available CPUs. • This assignment is carried out by software known as a scheduler and dispatcher.

16. Scheduling • Why is knowing scheduling important? • Multiple threads share the same process resources • Scheduler schedules them according to “some” algorithm • The sequence of scheduling can create conflicts • Read-write conflicts • Deadlocks (see slide 29) Multiple threads sharing a single CPU

17. Synchronization Conflicts • threaded code different from non-threaded? • protection for data shared among threads • synchronization among threads • Example Scenario: • ThreadA reads DataA • ThreadB reads DataA • ThreadA writes to DataA • ThreadB writes to DataA • The data written by ThreadA is lost

18. Synchronization Conflicts IntBalance = 5000; ThreadA() { Balance = Balance + 3000; } ThreadB() { Balance=Balance+(Balance *0.04); }

19. Process1 Process2 Balance = 5000 TL Order of Events interest 4% deposit 3000 TL deposit 3000 TL interest 4%

20. Synchronization Conflicts • Another example: • 3 Threads : • ThreadA : Produces data and puts it into a queue • ThreadB : Checks the queue and if there is a data removes and consumes it • ThreadC : Checks the queue and if there is a data removes and consumes it • How can things go wrong in this example?

21. Synchronization Conflicts IntQueuedataQ(); ThreadA() { Data d = ProduceData(); dataQ.enqueue(d); } ThreadB and ThreadC() { if(!dataQ.isempty()) { Data d = dataQ.dequeue(); ConsumeData(d) } }

22. Synchronization Conflicts • Problematic Scheduling : • Only 1 data left in queue • ThreadB checks for empty and receives false and enters the if-statement • ThreadCchecks for empty and receives false and enters the if-statement • ThreadCdequeues and consumes • ThreadB tries to dequeue an empty queue and crashes/error/exception

23. Synchonization • To prevent data conflicts additional structures are needed • Mutex • Semaphores • For C++ context we will focus on “mutex”

24. Mutex • A mutex (short for mutual exclusion) is a way of communicating among threads or processes that are executing asynchronously of one another. • This communication is usually used to coordinate the activities of multiple threads or processes, typically by controlling access to a shared resource by "locking" and "unlocking" the resource. • Source : http://msdn.microsoft.com/en-us/library/z3x8b09y(v=vs.71).aspx

25. Mutex • So a mutex is like a binary counter • To lock a mutex you “up” its value • To release a mutex you “down” its value • Once a mutex is up only a down operation is allowed • Use this mechanism to create invisible code barriers to prevent problems like before we encountered

26. Solving Problem 1 IntBalance = 5000; Mutexmdata; ThreadA() { up(mdata) Balance = Balance + 3000; down(mdata) } ThreadB() { up(mdata) Balance=Balance+ (Balance *0.04); down(mdata) }

27. Solving Problem 1 If Task 2 starts to enter the method, and Task 1 is already in the method, Task 2 is blocked until Task 1 finishes the method.

28. Solving Problem 2 IntQueuedataQ(); Mutexmqueue; ThreadA(){ up(mqueue) Data d = ProduceData(); dataQ.enqueue(d); down(mqueue) } ThreadB and ThreadC(){ up(mqueue) if(!dataQ.isempty()){ Data d = dataQ.dequeue(); ConsumeData(d) } down(mqueue) }

29. Deadlock Example

30. Multithread Programming • There is no set standard in C++ for Multithread programming ( Edit:2012 There is one now) • C++11 Standard • Every library or framework implements threads in a different way • Pthreads for *NIX • Windows Threads • Higher Level Libraries: • Boost:thread • Intel’s Thread Building blocks • OpenMP • Ting : ThreadING (Multiplatfrom library win/linux/OSX/Android) • Every OS has a different thread implementation. • Today we will focus on Windows threads

31. Multithread Programming • Two types of threads from a programmers point of view • Detached threads : Threads that spawn run and get destroyed after they complete their task • Joinable threads: Threads that are expected to join the parent thread after finishing the execution. These threads are not automatically destroyed.

32. Threads in Windows • Basic functionalities • Creating Threads • Suspending/Resuming Threads • Terminating/Exiting Threads • Synchronizing Execution of Multiple Threads

33. CreateThread • CreateThread function creates a new thread for a process • Returns an handle to the new thread • Syntax HANDLE CreateThread( LPSECURITY_ATTRIBUTES lpThreadAttributes, // pointer to security attributes DWORD dwStackSize,// initial thread stack size LPTHREAD_START_ROUTINE lpStartAddress, // pointer to thread function LPVOID lpParameter,// argument for new thread DWORD dwCreationFlags, // creation flags LPDWORD lpThreadId );// pointer to receive thread ID

34. Thread Start Address • Creating thread must specify the starting address of the code that the new thread will execute • Thread starting address is the name of the function defined in the program code • A process can have multiple threads simultaneously executing the same function

35. Thread Function Argument • It is risky to pass the address of a local variable if the creating thread exits before the new thread • Instead, either pass a pointer to dynamically allocated memory, or • Make the creating thread wait for the new thread to terminate

36. Thread Function Argument • Data can also be passed from the creating thread to the new thread using global variables • With global variables, it is usually necessary to synchronize access by multiple threads • In processes where a thread might create multiple threads to execute the same code, it is inconvenient to use global variables

37. CreateThread e.g. HANDLEhThread; intidThread; hThread = CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)MyTreadFunction, //Thread Function NULL, //parameter to be passed to the thread 0, (LPDWORD)&idThread); //Thread ID parameter

38. Thread Handles & Identifiers • A thread can use the • GetCurrentThreadId function to get its own thread identifier DWORD GetCurrentThreadId(); • GetCurrentThread function to retrieve a pseudo handle to its own threadhandle HANDLE GetCurrentThread();

39. Suspending Thread Execution • SuspendThread function suspends the specified thread • If the function succeeds, the return value is the thread's previous suspend count; otherwise, it is 0xFFFFFFFF • Syntax DWORD SuspendThread( HANDLEhThread);// handle to the thread

40. Suspending Thread Execution • Sleep function suspends the execution of the current thread for a specified interval • Syntax VOID Sleep(DWORDdwMilliseconds); // sleep time in milliseconds

41. Suspending Thread Execution • SwitchToThread function causes the calling thread to yield execution to another thread that is ready to run on the current processor • OS selects the thread to yield to • Syntax BOOL SwitchToThread();

42. Resuming Thread Execution • ResumeThreaddecrements a thread's suspend count • When the suspend count is decremented to zero, the execution of the thread is resumed • If the function succeeds, the return value is the thread's previous suspend count; otherwise, it is 0xFFFFFFFF • Syntax DWORD ResumeThread(HANDLE hThread); // identifies thread to restart

43. Terminating Threads • A thread executes until one of the following events occurs: • The thread function returns • The thread calls the ExitThreadfunction • Any thread of the process calls the ExitProcess function • Any thread calls the TerminateThread function with a handle to the thread • Any thread calls the TerminateProcess function with a handle to the process

44. ExitThread • The preferred method of exiting a thread • The thread's stack is deallocated • The entry-point function of all attached dynamic-link libraries (DLLs) is invoked with a value indicating that the thread is detaching from the DLL • Syntax VOID ExitThread( DWORD dwExitCode);

45. TerminateThread • Should be used only in extreme circumstances • The target thread has no chance to execute any code and its initial stack is not deallocated • DLLs attached to the targetthread are not notified that the thread is terminating • Syntax BOOL TerminateThread( HANDLEhThread, // handle to the thread DWORDdwExitCode); // exit code for the thread

46. Mutex Operations • Mutex Operations for windows in C++ are • Create a mutex • Aquire the mutex • Release the mutex • Destroy the mutex

47. Create a Mutex • CreateMutexCreates or opens a named or unnamed mutex object. • Syntax HANDLE WINAPI CreateMutex( __in_optLPSECURITY_ATTRIBUTES lpMutexAttributes, __in      BOOL bInitialOwner, __in_opt  LPCTSTR lpName); Source : http://msdn.microsoft.com/en-us/library/ms682411(v=vs.85).aspx

48. Aquire a Mutex • WaitForSingleObjectWaits until the specified object is in the signaled state or the time-out interval elapses. • To enter an alertable wait state, use the WaitForSingleObjectfunction. To wait for multiple objects, use the WaitForMultipleObjects. • Syntax • DWORD WINAPI WaitForSingleObject( __in  HANDLE hHandle, __in  DWORD dwMilliseconds ); Source: http://msdn.microsoft.com/en-us/library/ms687032%28v=vs.85%29.aspx

49. Release a Mutex • ReleaseMutex Releases ownership of the specified mutex object. • Syntax • BOOL WINAPI ReleaseMutex( __in  HANDLE hMutex ); Source: http://msdn.microsoft.com/en-us/library/ms685066%28v=VS.85%29.aspx

50. Demo • Let’s see all of that in action …