Microsoft Research: Future Possibilities In Concurrency

1. Microsoft Research: Future Possibilities In Concurrency Tim Harris FUN323Researcher MSR Cambridge

2. Concurrency Today And Tomorrow • Advances in computer hardware have always contributed to improved application performance • CPU speed, network bandwidth, storage capacities, graphics cards… • Now the development of multi-core CPUs provides a fresh opportunity • What tools and techniques are available from Microsoft to exploit this opportunity? • How is concurrency supported and used in the Microsoft platform? • What techniques is Microsoft investigating for programming future highly-parallel hardware?

3. ExampleDouble-ended queue 3 11 14 12 6 30 5 67 PushLeft / PopLeft work on the left hand end PushRight / PopRight work on the right hand end • Make it thread-safe by adding a lock • Needlessly serializes operations • One lock for each end? • Much more complex • Risk of deadlock when queue nearly empty • Still not composable • Still won’t scale up

4. Tools for detecting concurrency bugs at run-time New languages features as alternatives to locks Three Research Strands Language extensions to ensure disciplined locking

5. Tools For Detecting Concurrency Bugs At Run-Time

6. RaceTrackRun-time race detection • Run an existing application on the RaceTrack-modified runtime • Monitors all the managed code executed (including all managed libraries) • No program annotation is needed • No preprocessing is needed • RaceTrack gathers evidence of inconsistent lock usage • Warnings report potential races on-the-fly… • …even if the race hasn’t bitten in the monitored run

7. A B C B C D D A D C C Basic Algorithm • Idea: try to infer the locking discipline used by the program Warning: value’s lock-set has become empty T1 T2 T3 Lock candidates: o.next = o2; value: ? next: ? o.value = 1; o.value = 2; o.value = 3; Warning provides T3’s stack trace and details of the field being accessed

8. ExampleRaceTrack Warning POTENTIAL RACE [cac]: STFLD object 00AEA708 field_offset 72 0: System.Runtime.Remoting.Metadata.RemotingMethodCachedData.get_ReturnType c:\EverettSP\CLR\src\BCL\System\Runtime\Remoting\RemotingAttributes.cs: line 414 column 21 1: System.Runtime.Remoting.Proxies.RealProxy.PropagateOutParameters c:\EverettSP\CLR\src\BCL\System\Runtime\Remoting\RealProxy.cs: line 499 column 17 2: System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage c:\EverettSP\CLR\src\BCL\System\Runtime\Remoting\RealProxy.cs: line 406 column 25 3: System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke c:\EverettSP\CLR\src\BCL\System\Runtime\Remoting\RealProxy.cs: line 833 column 13 4: System.AppDomain.CreateInstanceAndUnwrap c:\EverettSP\CLR\src\BCL\System\AppDomain.cs: line 1690 column 13 5: AppDomainCSThd1MDH1.MakeDomain 6: AppDomainCSThd1MDH1.ThreadWork

9. use of “flags” in another method no locks on these methods line 414 ExampleRemotingAttributes.cs class RemotingMethodCachedData { bool CalcOneWay() { ... expensive ... } Type CalcReturnType() { ... expensive ... } enum Flags { CheckedOneWay = 0x01, ResultOneWay = 0x02, CheckedReturnType = 0x20 } Flags flags = 0; Type resultReturnType; T3 calls IsOneWay, sees CheckedOneWay is set bool IsOneWay() { if ((flags & CheckedOneWay) == 0) { Flags tmp = CheckedOneWay; if (CalcOneWay()) tmp |= ResultOneWay; flags |= tmp; return (tmp & ResultOneWay) != 0; } return (flags & ResultOneWay) != 0; } T2 T3 T3 continues, mistakenly sees ResultOneWay clear T1 T2 calls IsOneWay, sets flags to (ResultOneWay | CheckedOneWay) Type ReturnType { get { if ((flags & CheckedReturnType) == 0) { resultReturnType = CalcReturnType(); flags |= CheckedReturnType; } return resultReturnType; } } T1 calls get_ReturnType, preempted just before setting flags to CheckedReturnType T1 continues, stores CheckedReturnType in flags

10. Avoiding False Positives T1 • Divide threads’ execution into chunks • Only report races between chunks that may be concurrent • B&D may be concurrent • B&E cannot be concurrent A Thread.Start(…) T2 B D T3 Thread.Start(…) C E

11. Language Extensions To Ensure Disciplined Locking

12. Disciplined Locking • Today code doesn’t indicate what locks protect what data • The Spec# research language introduces design by contract features to document: • Object invariants and pre- and post-conditions on methods • The structure of aggregate objects • Which locks protect which data: a thread must own the data before it can touch it • Result: we can still reason as if the program was single-threaded when acting on shared data

13. node-a T1 ExampleShared Stack • References typed as owner/non-owner • This partitions the heap into thread-local regions and un-owned shared structures • Locking grants access to objects • Exclusive locking of a whole aggregate structure, or fine-grained read locking of constituent objects stack node-b node-c node-d Ordinary, non-owning, reference Owning reference: target object part of the same data structure

14. ExampleShared Counter class Counter { int count; public int Count{ … } public void Inc() requires owner == CurrentThread; ensures Count == old(Count) + 1 { count = count + 1; } public void AtomicIncBy2() { acquire (this) { int t = Count; Inc(); Inc(); assert (t==Count+2); } } } Counter ct = new Counter(); Heap.Share(ct); new Thread(new ThreadStart(ct.AtomicIncBy2)).Start(); new Thread(new ThreadStart(ct.AtomicIncBy2)).Start(); Require that the caller has acquired the counter object Calling Inc is guaranteed to advance the count by 1 Once it has acquired the counter, the thread can depend on having exclusive access to it Objects are initially owned by their allocating thread until they are shared

15. New Languages Features As Alternatives To Locks

16. Transactional Memory • Same programming idea as System.Transactions • But taking it further here to allow existing shared-memory operations to be performed atomically • Runtime system is responsible for • Making concurrent atomic blocks thread safe • Enabling scalability: • If two atomic blocks work on disjoint data then they can run entirely in parallel atomic { A.withdraw(3) B.deposit(3) } All of the operations performed in these methods look like a single atomic update to the heap

17. X 10 20 30 X T2: PopRight() Updates: queue.tail=node-d node_d.next=null node_d.value=X Reads: queue.tail node-e.prev node-d.value ExampleDe-queueing an item node_a node_b node_c node_d node_e queue Read Updated by T1 Updated by T2 T1: PopLeft() Updates: queue.head=node-b node_b.prev=null node_b.value=X Reads: queue.head node-a.next node-b.value

18. How Fast Is It? 100 Coarse-grained locking scales poorly because all operations are serialized 90 Fine-grained locking is fastest with few CPUs, but needs to be tuned to a specific workload 80 70 60 Time per operation /µs 50 40 30 Atomic blocks remain scalable because few operations conflict at run time 20 10 0 0 10 20 30 40 50 60 # CPUs • Threads using a shared hashtable • 16% updates, keys 1..4096

19. Waiting And Synchronization • retry means “we can’t continue yet: try again from the start of the atomic block” • Implementation avoids spin-waiting • The transaction log records the locations we’ve read from – wait until one of them is updated • No condition variables: no lost wake-ups Item WaitForLeft(Queue q) { Item i = q.PopLeft(); if (i == null) { retry; } return i }

20. Choice atomic {Item i;attempt { i = q1.PopLeft(3)} else { i = q2.PopLeft (3)} // Process item } A server thread is waiting on two clients: atomic {attempt { WaitForShutdownRequest(); // Process shutdown request } else { } } Unlike WaitAny we can compose code using retry – e.g. wait for a shutdown request, or for client requests Try this first – if it gets an item without hitting retry then we’re done If q1.PopLeft can’t finish then undo its side effects and try q2.PopLeft instead

21. Synchronization Using Chords • The Cω research language adds a keyword to identify asynchronousmethods • Decouples the execution of caller and callee • Chords defined over sets of methods provide a general synchronization mechanism ‘put’ is an asynchronous method: callers can return immediately and the runtime system queues up the call class Buffer { async put(String s); String get() & put(String s) { return s; } } ‘get’ and ‘put’ are combined in a chord whose body ‘plays’ when calls have been made to both methods

22. Service1 Service2 Client ExampleAsynchronous Requests public class Service { public async request(String arg, IntCB callback) { // compute result callback(result); } } public delegate async IntCB(int v); Clients make asynchronous calls on a service to issue requests Clients supply a callback delegate which the server invokes when it’s done

23. ExampleJoining Responses class Join2 { async first(int r); async second(int r); void join(out int cr1, out int cr2) & first(int r1) & second(int r2) { cr1 = r1; cr2 = r2; return; } } Join2 x = new Join2(); service1.request(arg1, new IntCB(x.first)); service2.request(arg2, new IntCB(x.second)); // do something useful in the meantime // now wait until both results have come back int r1, r2; sum = x.join(out r1, out r2); // do something with results Service requests are async: caller can get on with other work Caller synchronously waits for the results to come in when they are needed

24. Conclusion

25. Conclusion • Three research strands in this talk: • Testing for races with RaceTrack • Preventing race conditions with Spec# • Controlling concurrency with atomic blocks and synchronization chords • These projects have looked at concurrency based mainly on explicit threading • Other research projects are investigating: • Data parallel languages • Parallel evaluation of side-effect-free functional code

26. Community Resources • At PDC • Watch the DVD recordings of • FUN302: Programming with Concurrency (Part 1): Concepts, Patterns, and Best Practices • FUN405: Programming with Concurrency (Part 2): Multithreaded Programming with Shared Memory • To learn about System.Transactions watch • FUN320: Windows Vista and “Longhorn” Server: Improving Reliability Using System.Transactions and the Transactional NTFS and Registry • And later this morning see • TLN309: C++: Future Directions in Language Innovation (Fri 10:30am)

27. Community Resources • RaceTrack • Contact us for prototype availability • http://research.microsoft.com/research/sv/racetrack • Spec# • Alpha prototypes available for download • http://research.microsoft.com/SpecSharp/ • Atomic blocks • Prototypes available for Haskell and C# (as a library) • Contact tharris@microsoft.com • Synchronization chords • Prototype available as part of Cω • http://research.microsoft.com/comega/

28. © 2005 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.