Guide to multithreaded code

1. Guide to multithreaded code “All I need to know about writing multithreaded code is that I am not smart enough to do it right.” - Terry Way Intended audience: • C/C++ programmers not already expert in threading. • Visual Basic and Java programmers - it’s not all done for you! • Managers - see the issues that must be addressed. • Part I Overview of the issues - Managers and developers • Part II Coding issues - Developers A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

2. Part I • An overview for managers and developers • Performance issues are marked with a red arrow. • Most of the other issues relate to reliability. • Best practice advice is marked with a blue tick. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

3. Part II • Coding issues, mostly for developers • Support within COM (limited amount of COM+) • Support within Visual Basic • Support within Java • How to use C++ effectively • Avoiding reliability problems • Tackling performance problems A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

4. Common myths • Threads will make the system run faster. • We don’t need to test on SMP machines. • The chances of two threads doing X are so small. • Only C/C++ programmers need concern themselves. • We can bother about that later. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

5. The right tools • SMP machines • Appropriate libraries:Dinkumware STL www.dinkumware.com Smartheap www.microquill.com Hoard www.cs.utexas.edu/users/emery/hoard/ Threads.h++ www.roguewave.com • Configure msvcrt optimally www.wdj.com/archive/1105/feature.html MSVCRT_HEAP_SELECT A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

6. Why use SMP? • You may not be targeting SMP machines, but you still need to test with SMP machines. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

7. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

8. Why test on SMP? • There are categories of bugs that almost never occur on a uniprocessor machine, but are much more likely to occur on an SMP machine. Your choose is: • Test on a uniprocessor machine with a very varied data set, and on all CPU speeds that you will ever run on (or vary the quantum size from 1 to 100ms). • Let your most important customer find the bugs, because he has the most unusual data set. • Test on an SMP machine. • Which costs most now? A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

9. You’re exaggerating right? • Yes and No. • In 25ms a huge amount of instructions can be performed. So the chance of a thread swap occurring in the critical place is small. • This is a demonstration of a bug that is almost impossible to test for on a uniprocessor machine. Other more common classes of bug are easier to test for without SMP, and but much easier to test for on an SMP machine. You reduce the number of test case iterations to find a bug with SMP. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

10. SMP benefits • Reduced testing and/or more confidence. • Scalability (for servers). • More horse power. You can buy a 2 x 1GHz machine for less than a 1.4GHz one way machine. • Motivation - put a smile on a developer’s face and upgrade his machine. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

11. SMP downsides • Not everybody’s software runs on SMP. • Hey, this just proves my point, SMP finds bugs for you! • Badly written code can actually run slower on SMP. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

12. Why use threads? • Wait on a blocking call without hanging the UI thread. • Smooth scheduling. • Apparent performance. • Division of truly independent and parallel tasks (e.g. background tasks, or multiple clients) • Avoid poling.Poling is almost never right. Pole too often and you burn CPU power, too slow and you appear unresponsive. Also you can completely miss events. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

13. Threads have an overhead • Every thread that you create has an overhead. It needs • kernel resources to manage it • a stack (memory, with a typical reserved size of 1MB) • it needs scheduling • it needs creating • it needs to be cleaned up after it quits • OS handles must be duplicated • it can add complexity to the program • it may compete for resources • you may need to swap between threads • Only create a thread if there is a real need. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

14. KISS • Keep it simple stupid! • If you don’t have a justified requirement for multiple threads, then you are causing unnecessary complication. • This talk is non-trivial and so is threading. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

15. Advice • Illustrate the object relationships, clearly showing the tiers . • Document the threading model of your design. • Illustrate what objects reside in what threads or apartments. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

16. Terminology • SMP • Symmetrical multiprocessing. More than one CPU in a machine, each with equal access to system resources. CPU 0 CPU 1 on board cache cache sync on board cache external cache Main memory A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

17. Terminology • Serialize • To “order sequentially”. Often we need to serialize requests (function calls). This means that if two threads call a function foo() then we must ensure that the thread A has fully executed foo() before the thread B is permitted to start executing foo(). A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

18. Terminology • Thread safe lock (abbreviated to lock) • A resource that only one thread can hold at anyone time. thread_safe_lock_t my_lock; // Protects data below std::string my_string; std::vector<int> my_numbers; void foo() { my_lock.lock(); my_numbers.push_back(my_string.size()); my_lock.unlock(); } A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

19. How locks work A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

20. How locks work 1) Thread A enters the function and acquires the lock. 2) Thread A gets on with executing the function. 3) Thread B enters the function but cannot acquire the lock, so it waits until the lock is available. 4) Thread A unlocks the lock and leaves the function. 5) Thread B is now able to acquire the lock, and execute the function. 6) Thread B unlocks the lock and leaves the function. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

21. Overview of threading problems 1) Dead locks • This is where a thread is stops in its tracks waiting for some event to be signaled, or some resource to become free. But that will never happen. So the thread waits for ever. 2) Lack of atomic calls • It is possible for the object to be accessed whilst in an inconsistent state, because what is logically a single operation is split across more than one function call. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

22. Overview of threading problems 3) Race conditions • e.g. A sent event is generated by one thread and an acknowledgement by another thread. It may be possible to receive the acknowledgement before receiving the initial sent event. 4) Data corruption • e.g. String is copied before it is completely written. Similar to the SMP example. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

23. Overview of threading problems 5) Indirect freezes • The UI freezes because it is waiting on a resource held by a background thread that is performing blocking operation. 6) Lost locks • This is a special case of deadlock, where a lock has been acquired but the code forget to release the lock. The next thread to attempt the acquire the lock will wait for ever. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

24. Overview of threading problems 7) Resource contention • This is where there is a very frequently used resource that only one thread can use at a time. The CPU can spend more time swapping threads than it does doing the real work. 8) Programmed thread context swapping • Typically caused by a local/remote procedure call (lrpc) being made from thread A to thread B. This is almost as expensive as interprocess communication. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

25. Overview of threading problems 9) Mixed component capabilities • Some are thread safe and some are not 10) Mixed environment expectations • Visual Basic, C++, Java, COM, COM+ and third party solutions may all have differing threading expectations and solutions, creating confusion. 11) Every body has a solution • If I had $1 for every CThread class I’ve seen  A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

26. Tools • The programming environment • VB/Java/C++ COM COM+ • Debugger • Perfmon • QSlice (quick slice) • Profiler A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

27. Questions ? End of part I A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

28. Environment support • Windows NT/2000 - far too much to cover, and mostly too low level • COM • Visual Basic • Java • C++ A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

29. COM support • The problem: • Component A was written in VB and is not thread safe. • Component B was written in C++ or Java and is thread safe. • These components cannot safely interact. • Either the whole system must be written using one scheme or we must devise a way to interconnect them safely. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

30. COM support How COM solves the problem: A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

31. COM support How COM solves the problem: A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

32. Cross apartment calls Cross apartment calls are very expensive. A large amount of RPC code gets executed, and two thread swaps have to occur. One thread swap to execute the code in object B, and another to get back to A’s thread. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

33. COM - example 1 A dialog in the primary STA creates a line object. The line object creates two points. The line object is threading model “both”. The points are threading model “apartment”. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

34. COM example 2 The points cannot be instantiated in the MTA so a proxy is used. This is very inefficient. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

35. Management implications • In an ideal world: • all developers would be expert in all languages. • each component would be written in the most suitable language for the domain. • In the real world • most developers are proficient in just one or two languages. • resource and scheduling issues forces managers to choose the language that the component is written in based on the available skills. • Result • major inefficiencies can be built into a product. • if you are lucky and you find the performance bottle neck you may need to completely rewrite the component in another language. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

36. More inefficiencies A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

37. Free threaded marshaler A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

38. Free threaded marshaler A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

39. Free threaded marshaler A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

40. One Solution • NT 4.0 SP3 introduced the GIT (global interface table). You store a “handle to an object” rather than a direct pointer. Before accessing an object you can give the GIT the handle and ask it for a valid pointer (to a proxy). • GITPtr - see code handout. • If anyone wants to use this, I can talk about it separately. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

41. COM support summary • For all this work COM has only solved a couple of problems: * It has directly solved problem (9) mixed component capabilities. * Because it is a standard on Microsoft platforms it indirectly solves problem (10) mixed environment expectations. • However, it solves these problems at the cost of problem (8) thread context swapping. But not with thread neutral model. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

42. Enter COM+ • Windows 2000 only. • Much of COM+ is about transaction processing. • In addition to a “threading model” components can now have a “synchronization” attribute. • There is a finer grain of isolation of incompatible code, each apartment can have one or more contexts. • Thread neutral apartment. • I haven’t used it. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

43. Thread neutral apartment • COM+ only. • Now Microsoft’s recommended threading model for non-UI code. Seriously consider it when targeting Windows 2000. • Like FTM but without the problems. No need for GITPtr. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

44. COM+ synchronization • Disabled • Just like COM • Not Supported • Objects make no promises, it is not synchronized. (These ought to be apartment threaded.) • Supported • Objects provide their own (custom) synchronization. • Required • Requires COM+ to provide the synchronization. Even when it’s not needed. • Requires New • ? A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

45. Questions? • Next we’ll see how Visual Basic fits into the COM framework. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

46. Theme • A recurring theme in writing robust software is to restrict things as much as you can, please watch out for it. • Prefer private over public • Prefer variables in this order: • automatics, parameters, members, globals • Prefer const over non-const • The more you expose the more there is to go wrong. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

47. VB variables Sub foo() dim x as Integer x = Something End Sub • This variable does exactly what you expect. It has the life time of the function call. It is only visible within the function. • See handout “vb_rules”. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

48. VB variables Private m_credits_earned as Integer Sub Grade() If m_credits_earned > 50 Then call TopGrade End If End Sub • The code excerpt comes from a VB ‘student’ class. • This variable does exactly what you expect. The life time of m_credits_earned is the same as the lifetime of the student. Each separate student has a different instance of m_credits_earned. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

49. VB variables Sub Bar() static is_recursive as boolean If is_recursive Then Exit Sub End If is_recursive = true call DoStuff ‘ beware DoStuff can call Bar is_recursive = false End Sub • This use of static does exactly what you expect and want. Although it is rather different to the C++ meaning with the same syntax. It declares a member variable only visible to the function. Member variables have the lifetime of the form or class, or module they are part of. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001

50. VB variables Private g_total_something as Integer Sub Foo() If g_total_something < 10 Then call DoStuff g_total_something = g_total_something + 1 End If End Sub • The code excerpt comes from a VB module. The behavior is different for a form or class, and anyway you would use m_total_something in a form or class. • This variable does exactly what you expect and want until the code is run in a different apartment. The code in the new apartment uses a new instance of m_total_something and initializes it to zero. A practical guide to writing multithreaded code - Mark Bartosik Febuary 2001