Windows Application Development

1. Windows Application Development Chapter 5 Threading Models for Reliability

2. OBJECTIVES • Upon completion of this Chapter, you will be able to: • Solve more complex threading problems • Use standard threading models in application design • Combine mutexes and events reliably and usefully • Build reliable, maintainable applications • Understand and avoid many common pitfalls

3. Contents • 1. Multithreading Models • 2. COM Model Comparison • 3. Events and Mutexes Together • 4. The Condition Variable Model • 5. SignalObjectAndWait • 6. Using the Condition Variable Model • 7. Some Performance Tradeoffs • 8. Avoiding Incorrect Code - Hints • 10. Lab Exercise 5

4. 1. Multithreading Models • Advantages of using thread models: • Expedite design and development • Models are well understood and well tested • Avoid potential multithreading problems • Models match structure of most programming problems • Very simple cases, plus three “classic” models • Boss/worker • Pipeline • Client/server • Plus, combined models • “Programming in the large”

5. Files Databases Boss/Worker Model Input (Stream) Program Resources Workers Task1 Boss main () Task2 ... TaskN Computations

6. Pipeline Model Message Log P1 C1 TO FROM DATA M1 M2 . . . M5 Q Q Q CN PN Transmitter Receiver Q Producers Consumers

7. Client/Server Model Server Thread 1 Stack 1 TLS Client 1 Code Shared Data Constants Thread-Thread Communication & Synchronization Arg Thread 1 Specific Data TLS Status Broadcast and Monitor Server Thread N Stack Stack TLS Client N Arg Arg Thread N Specific Data Statistics

8. Combinations • Large systems rarely a pure instance of a single model • Pipeline stages often implemented as boss/ worker systems • A server may perform client requests in a pipeline sequence • Always use good implementation documenting individual modules in terms of these models • Easier to maintain, understand and troubleshoot

9. 2. COM • Microsoft’s Component Object Model • Object-oriented technology • Single threaded object • Only one thread can access it • Apartment model threading • Unique thread assigned to each instance of an object • Free threaded object • A thread from a thread pool is assigned, or created, when a request is made

10. COM – Useful Terminology • Thread pool • Collection of available threads • Symmetric thread model • A group of threads performs the same task using exactly the same thread functions • Asymmetric thread model • Different for separate thread functions • COM only • Don’t confuse with “Classical Models”

11. 3. Events and Mutexes Together • Example: Pipeline producer/consumer • Producer creates a checksummed message • A mutex protects the message • Producer signals the consumer that a message is ready • The producer waits on an event • The mutex defines the critical code section for accessing the message data structure object • Assures the object’s “invariant” properties • The event is used to signal that there is a new message • Signals that the object (message queue) is in a specified state

12. Events and Mutexes Together • One thread (producer) locks the data structure • Changes the object’s state by creating a new message • Sets or pulses the event – new message • One or more threads (consumers) wait on the event for the object to reach the desired state • The wait must occur outside the critical code section • A consumer thread can also lock the mutex • And test the object’s state • The state will not changer while being tested

13. 4. The Condition Variable Model • “Programming in the small” • Concepts are taken from Pthreads (Supplem’try Section) • Several key elements: • Data structure of type STATE_TYPE • Contains all the data such as messages, checksums, etc. • A mutex and one or more associated events • The mutex protects the data structure • The events signal “interesting” data structure state changes • Boolean functions evaluate “condition variable predicates” • For example, “a new message is ready” • An event is associated with each condition variable predicate • The event is signaled with the cvp becomes true

14. Condition Variable Model: Data Shared Data Structure: typedef struct _state_t { HANDLE Guard; /* Mutex to protect the object */ HANDLE cvpSet; /* Autoreset Event */ . . . other condition variables /* State structure with counts, etc. */ struct STATE_VAR_TYPE StateVar; } STATE_TYPE State; . . . /* Initialize State, creating mutex & event(s) */ . . .

15. Condition Variable Model: Producer /* PRODUCER thread that modifies State */ WaitForSingleObject (State.Guard, INFINITE); /* Change state so that the CV predicate holds */ . . . State.StateVar.xyz = . . . ; SetEvent (State.cvpSet); ReleaseMutex (State.Guard); /* End of the interesting part of the producer */ . . .

16. Condition Variable Model: Consumer /* CONSUMER thread waits for a particular state */ WaitForSingleObject (State.Guard, INFINITE); while (!cvp(&State)) { ReleaseMutex (State.Guard); WaitForSingleObject (State.CvpSet, TimeOut); WaitForSingleObject (State.Guard, INFINITE); } . . . ReleaseMutex (State.Guard); /* End of the interesting part of the consumer */

17. Condition Variable Model Comments • You need to repeat the loop and test • The timeout is finite – a tunable parameter • CVM avoids missed signals and other problems • This is the “broadcast” version • Multiple consumers released by one producer SetEvent() • Subsequent behavior depends on cvp() and produce/consume semantics • Uses an AR event and SetEvent() • “Signal” version uses MR event and PulseEvent() • A single consumer is released

18. 5. SignalObjectAndWait • Three essential steps of the loop in the consumer • Unlock the mutex • Wait on the event • Lock the mutex again • The first two are not atomic • The producer can signal before the consumer waits • Risk: A missed signal – hence the timeout • SignalObjectAndWait() combines the first two steps • One atomic function • Pthreads combines all three steps • >= NT 4.0 (not on Win 9x, Me)

19. SignalObjectAndWait • DWORD SignalObjectAndWait( • HANDLE hObjectToSignal, // Mutex • HANDLE hObjectToWaitOn, // Event • DWORD dwMilliseconds, // time-out in ms • BOOL bAlertable // Use FALSE – until S7; • SOAW is general purpose • Just the mutex-event usage is shown • Timeout can usually be infinite • Be sure to: • #define _WIN32_WINNT 0x400 // WINBASE.H

20. 6. Using the CV Model • Producer locks the mutex, changes state, sets the event and unlocks the mutex • Event should be set with the mutex locked • Consumer tests the CV predicate with the mutex locked • If the predicate does not hold, the consumer must unlock the mutex before waiting on the event • Event wait must have a timeout – to avoid missed signals • Unless you use SOAW! • Consumer always retests the predicate after the event wait • Consumer always owns the mutex when it leaves the loop • Whether loop body was executed or not

21. CV Model Variation • In producer/consumer code, multiple threads released • Auto-reset event • SetEvent • Or, there may be only one message available and multiple consuming threads • Event should be manual-reset • Producer should call PulseEvent • Assure exactly one thread is released

22. When Interested in the Next Event /* CONSUMER thread waits for NEXT state change */ WaitForSingleObject (State.Guard, INFINITE); do { SignalObjectAndWait (State.Guard, State.cvpSet, TimeOut, FALSE); WaitForSingleObject (State.Guard, INFINITE); } while (!cvp(&State)); /* Thread now owns the mutex and cvp(&State) holds */ /* Take appropriate action, perhaps modifying State */ . . . ReleaseMutex (State.Guard); /* End of the interesting part of the consumer */

23. 7. Some Performance Tradeoffs

24. 8. Avoiding Incorrect Code - Hints • Pay attention to design, implementation, and use of familiar programming models • Best debugging technique: Don’t create bugs in the first place • Many serious defects will elude the most extensive and expensive testing • Debuggers change timing behavior • Masking the race conditions you wish to expose Most of these hints are from Programming with POSIX Threads by David Butenhof, Addison-Wesley, 1997.

25. Avoiding Incorrect Code - Hints • Avoid relying on “thread inertia” • Never bet on a thread race • Scheduling is not the same as synchronization • Sequence races can occur • Even when you use mutexes to protect shared data • Cooperate to avoid deadlocks • Never share events between predicates

26. Avoiding Incorrect Code - Hints • Beware of sharing stacks and related memory corrupters • Beware: No warning of thread stack overflow! • Be sure to use the volatile storage modifier • Use the condition variable model properly • Understand your invariants and condition variable predicates • Keep it simple • Test on multiple systems (single and multiprocessor) • Testing is necessary but not sufficient • Be humble • Be prepared for unpleasant surprises

27. Thread Blocking Notes • At any point any thread may go to sleep for an unbounded period of time • No ordering exists between threads unless you cause ordering • Thread scheduling behavior can vary widely • Various operating system implementations • Different models of the same system • Systems with different processor speeds • Explicit synchronization is required • There is no timeout from a pthread_join() call

28. Word Tearing • A single mutex should be used in every critical section associated with a shared resource • a.k.a. “fighting over cache lines” - SMP • Example: A processor may access quadword memory units • 64 bits, since a word is 16 bits • 16-bit words are not separate resources • When different threads access distinct values that lie in the same quadword, we have word tearing • The entire quadword should be protected with one mutex • Demonstration program included in exercise files • Ask the instructor for more information if interested!

29. 11 11 11 11 11 51 22 22 22 22 22 22 33 33 33 33 33 33 44 44 48 44 44 48 Word Tearing (2 of 2) Memory 4 16- Bit Words Thread A Thread B Read Read Writes to First Byte Writes to Last Byte Terminates Terminates Thread B Value Wins; Next Time; Thread A Might Win

30. Deadlock Avoidance, Lock Hierarchies • Avoid deadlocks by using a lock hierarchy whereby mutexes are always locked in the same sequence • Release locks in the reverse sequence • Additional deadlock notes • During development and troubleshooting • Assure that multiple locks are always acquired in the same order • Document the lock hierarchy • Lock hierarchies are provably deadlock free

31. No Deadlocks with Hierarchical Locks AddSharedElement Thread OS SCHEDULER DeleteSharedElement Thread Running Ready Lock (&ListA.guard) Preempt Run Running Ready Blocked Lock (&ListA.guard) Run Lock (&ListB.guard) Running Blocked Unock (&ListB.guard) Unlock (&ListA.guard) Unblock Ready Run Preempt Lock (&ListB.guard) Ready Running Unock (&ListB.guard) Unlock (&ListA.guard)

32. 9. Lab Exercise 10-1 • ThreeStage.c implements the illustrated pipeline model • See the Pipeline slide • It was used to generate the performance results • It is implemented with SignalObjectAndWait • Fix QueueObjX.c, and/or • Determine performance of different implementations on your system(s)