E81 CSE 532S: Advanced Multi-Paradigm Software Development

1. E81 CSE 532S: Advanced Multi-Paradigm Software Development Synchronization Patterns Christopher Gill, Todd Sproull, Eric DeMello Department of Computer Science and Engineering Washington University, St. Louis cdgill@cse.wustl.edu

2. An Illustrative Haiku Threads considered bad. So non-deterministic. What will happen nxte? - Justin Wilson, Magdalena Cassel, Adam Drescher, Chris Gill

3. Design for Multithreaded Programming • Concurrency • Logical (single processor): instruction interleaving • Physical (multi-processor): parallel execution • Safety • Threads must not corrupt objects or resources • More generally, bad inter-leavings must be avoided • Atomic: runs to completion without being preempted • Granularity at which operations are atomic matters • Liveness • Progress must be made (deadlock is avoided) • Goal: full utilization (something is always running)

4. Multi-Threaded Design, Continued • Benefits • Performance • Still make progress if one thread blocks (e.g., for I/O) • Preemption • Higher priority threads preempt lower-priority ones • Drawbacks • Object state corruption due to race conditions • Resource contention (overhead, latency costs) • Need isolation of inter-dependent operations • For concurrency, synchronization patterns do this • At a cost of reducing concurrency somewhat • And at a greater risk of deadlock

5. Multi-Threaded Design, Continued • Race conditions (threads racing for access) • Two or more threads access an object/resource • The interleaving of their statements matters • Some inter-leavings have bad consequences • Example (critical sections) • Object has two variables x Є {A,C}, y Є {B,D} • Allowed states of the object are AB or CD • Assume each write is atomic, but writing both is not • Thread t writes x = A; and is then preempted • Thread u writes x = C; y = D; and blocks • Thread t writes y = B; • Object is left in an inconsistent state, CB

6. Multi-Threaded Programming, Continued • Deadlock • One or more threads access an object/resource • Access to the resource is serialized • Chain of accesses leads to mutual blocking • Single-threaded example (“self-deadlock”) • A thread acquires then tries to reacquire same lock • If lock is not recursive thread blocks itself • Two thread example (“deadly embrace”) • Thread t acquires lock j, thread u acquires lock k • Thread t tries to acquire lock k, blocks • Thread u tries to acquire lock j, blocks

7. Synchronization Patterns • Scoped Locking (via the C++ RAII Idiom) • Ensures a lock is acquired/released in a scope • Thread-Safe Interface • Reduce internal locking overhead • Avoid self-deadlock • Strategized Locking • Customize locks for safety, liveness, optimization • These complement a number of concurrency patterns that we’ll cover as well over time

8. Scoped Locking Pattern • Intent • Ensure lock is acquired when control enters a scope and is released automatically when control leaves, by any path • Context • Code that should not execute concurrently, should be protected (made atomic) by a lock • However it is hard to ensure that locks are released in all paths through the code • C++ code can leave a scope due to a return, break, continue, or goto statement, or a propagating exception

9. Scoped Locking, Continued • Solution • Define a guard whose constructor automatically acquires a lock when control enters a scope • Destructor automatically releases the lock when it leaves the scope // from Williams pp. 38 void add_to_list (int new_value) { // RAII (a.k.a. guard idiom) std::lock_guard<std::mutex> guard (some_mutex); // may throw an exception (e.g., bad_alloc) some_list.push_back(new_value); }

10. Thread-Safe Interface Pattern • Intent • Minimizes locking overhead • Ensures intra-component method calls do not ‘self-deadlock’ • Context • Intra-Component method calls • public methods (accessible from outside a class) • private implementations which change component state • Recursive mutex: higher overhead • Non-recursive mutex: risk of deadlock

11. Thread-Safe Interface, Continued public: void init () { std::lock_guard<std::mutex> guard (my_mutex); // ... does something inner_call (); } void foo () { std::lock_guard<std::mutex> guard (my_mutex); // ... does something else inner_call (); } void inner_call () { // ... does not take a lock } • Solution • Separate locking from implementation • Encapsulate acquire/release within public interface methods • “at the border” • Encapsulate implementation in private methods • Do not acquire/release • Important restriction: do not “call up” to public interface methods

12. Thread-Safe Interface, Continued • Variant: thread-safe façades and wrapper façades • Synchronize an entire subsystem or API • Calls (e.g., into OS kernel) may block until completion • Benefits • Helps prevent Intra-Component-Incurred-Self-Deadlock • Use std::unique_lock and std::adopt_lock to transfer lock ownership • Use std::lock function to grab >1 mutexes at once • See Williams Chapter 3.2 for more on these issues • Helps avoid unnecessary acquire/release calls • Allows addition of thread-safe wrappers to legacy code

13. Strategized Locking Pattern • Intent • Parameterizes synchronization mechanisms that protect a component’s critical section from concurrent access • Context • Components can be re-used efficiently within a variety of different concurrent applications • Different applications might need different synchronization strategies • Mutex • Readers/Writer locks • Semaphores • Solution • Parameterize code with lock type to decouple them • Can modify locks without changing application logic • Can modify application w/o changing lock implementations

14. Strategized Locking, Continued • Solution illustrated • Parameterized synchronization protects critical sections • Can plug in std::mutex or std::recursive_mutex or your own readers-writer lock or null lock, etc. as needed template <class LOCK> class File_Cache { public: const void *access (const string &path) { std::lock_guard<LOCK> guard (lock_); //... implementation of the access method } private: LOCK lock_; };

15. One More Design Issue: Single Initialization • Double Checked Locking Optimization etc. have limitations • Generally speaking, hard to eliminate data races without interface design • C++11 introduces helpful std::once_flag and std::call_once • Each thread calls std::call_once • Initialization guaranteed before the call returns // from Williams pp. 61 void foo () { // call_once protects call to init_resource std::call_once(resource_flag,init_resource); // thread-safe because resource is initialized resource_ptr->do_something(); }