CCR - Concurrency and Coordination Runtime

1. Andreas Ulbrich (anulb@microsoft.com) SDE – Microsoft Robotics CCR -Concurrency and CoordinationRuntime

2. Overview • Why CCR? • Hello, World! • Message-Based Coordination • CCR Examples • Asynchronous Programming Model (APM) • User Interfaces • Error Handling

3. Why CCR? • Concurrency • Process many task simultaneously • Scalability, Responsiveness • Leverage parallelism • Coordination • Exercise control • Orchestrate asynchronous operations • Handle (partial) failures • Runtime • Scheduler, Extensibility

4. CCR Programming Model • Asynchronous message passing (in-process) • No explicit threads, locks, semaphores! • Task scheduled based on message availability • Data-dependency scheduler • Models concurrency • Coordination primitives (join, choice, …) • Composition of data-driven components • Iterative tasks • Express sequential control flow of async. tasks

5. Hello, World! Port: channel for sending and receiving messages var queue = new DispatcherQueue(); var port = new Port<string>(); port.Post("Hello, World!"); Arbiter.Activate(queue, Arbiter.Receive( false, port, message => Console.WriteLine(message) ) ); Task queue: dispatcher schedules queues RR, task must be activated on a queue Receive arbiter: coordination primitive Post: sends a message Not persistent: handles only one message Port on which to receive the message Task: delegate that handles the message (message is consumed)

6. Dispatcher schedules items from its queues round-robin to run in its threads. Post places a message on the port Enqueue work item Port Dispatcher Scheduler picks next work item to execute Arbiter is activated on queue Thread calls handler with message as arg. Creates work item from message and handler Arbiter checks whether it can consume the message. Arbiter DispatcherQueues Threads Handler Arbiter is attached to port

7. There can be many of everything Port Dispatcher Arbiter DispatcherQueues Threads Handler Handler Handler

8. Using Messages for Coordination • Message triggers an operation • PortSet<Operation1, Operation2> • Message carries parameter for operation • Decouples sender from actual implementation • Message signals completion of operation • Port<Result>, PortSet<Result, Exception>, … • Port is stand-in for actual result • Port can be send to other tasks • Coordinate tasks with messages

9. Coordination Primitives Task-Scheduling (non port-specific) Single-Port Primitives Multi-Port Primitives FromHandler Single Item Receiver Join (Logical AND) FromIterator Handler Multi-Item Receiver Choice (Logical OR) Multi-Port Receiver Interleave (Reader/Writer)

10. Choice • Executes at most one of its child-tasks PortSet<string, Exception> resultPort = … Arbiter.Activate( queue, Arbiter.Choice( resultPort, result => Console.WriteLine("result: " + result), exception => Console.WriteLine("exception“) ) ); Handler if string received Handler if exception received

11. Joined Receive • Executes when all of its branches can execute • Coordinate on completion of concurrent tasks • Atomic consumption to prevent deadlocks Arbiter.Activate(queue, Arbiter.JoinedReceive<string, string>(false, resultPort1, resultPort2, (result1, result2) => { Console.WriteLine(“done”); } ) ); Ports on which to receive messages Handler receives both results as arguments

12. Multi-Item/Multi-Port Receive • Executes on reception of multiple items of the same type • Scatter/Gather, coordinate completion of parallel tasks • Atomic consumption to prevent deadlocks Arbiter.Activate( queue, Arbiter.MultiplePortReceive( false, resultPorts, results => { Console.WriteLine(results.Length); } ) ); Array of ports on which to receive messages Handler receives array of results as argument

13. Overview • Why CCR? • Hello, World! • Message-Based Coordination • CCR Examples • Asynchronous Programming Model (APM) • User Interfaces • Error Handling

14. Asynchronous Programming • BCL: Asynchronous versions for many operations • BeginOperation, EndOperationpair • Callback when operation completed/failed • BeginRead(buffer, offset, count, callback, state) • Returns IAsyncResult (moniker for pending result) • Also passed to callback • EndRead(asyncResult) • Returns result of operation

15. APM Gotchas • Callback maybe called from any thread, e.g. • Calling thread (synchronous completion) • Thread pool (potential to starve) • Depends on implementation of Begin/End • Coordination is clumsy • Sequential (continuation passing, nested delegates) • Recurrence (ping pong between callbacks) • Scatter/Gather, Partial failure

16. APM with CCR • Use CCR to • Model concurrency of application • Coordinate asynchronous operations • Getting out of APM is easy • In the callback post IAsyncResult to a CCR port

17. Iterative task: models sequential control flow Begin read operation “wait” for result of read Process read data Asynchronous Read Use port to coordinate on completion of async. operation IEnumerator<ITask> CcrReadFileAsync(string file) { varresultPort = new Port<IAsyncResult>(); using (varfs= new FileStream(file,…,FileOptions.Asynchronous)) { varbuf= new byte[fs.Length]; fs.BeginRead(buf, 0, buf.Length, resultPort.Post, null); IAsyncResult result = null; yield return Arbiter.Receive(false, resultPort, ar=> { result = ar; }); try { fs.EndRead(result); ProcessData(buf); } catch { // handle exception } } } Callback: simply post IAsyncResult as a message 1 2 3 Yield receiver task, Task is activated by dispatcher, Dispatcher calls MoveNext() when task complete. Does not block thread!

18. Asynchronous Read IEnumerator<ITask> CcrReadFileAsync(string file) { varresultPort = new Port<IAsyncResult>(); using (varfs= new FileStream(file,…,FileOptions.Asynchronous)) { varbuf= new byte[fs.Length]; fs.BeginRead(buf, 0, buf.Length, resultPort.Post, null); yield return (Receiver)resultPort; var result = (IAsyncResult)resultPort; try { fs.EndRead(result); ProcessData(buf); } catch { // handle exception } } } Simplified notation

19. Activating Iterative Tasks • Directly • queue.Enqueue(new IterativeTask(“test.txt”, CcrReadFileAsync)); • Yielded from an iterative task • yield return new IterativeTask(…); • As task conditioned by an arbiter • Arbiter.Activate(queue,Arbiter.ReceiveWithIterator( false, port, CcrReadFileAsync ))

20. Benefits of Iterative Tasks • Simplified code • Access to locals, no need to pass async. state • Sequential control flow, e.g. loops • No nesting, ping/pong of callbacks • Ability to use using,try/finally • Simplifies resource management

21. Example: Stream Copy • Block-copy large file from input stream to output stream • Sequential control flow with repeated async. operations • while (input not empty): read block from input write block to output

22. CCR Stream Wrappers Port set: collection of ports, one for each possible result of the operation public static PortSet<int, Exception> Read( Stream stream, byte[] buffer, int offset, int count) { varresultPort = new PortSet<int, Exception>(); stream.BeginRead(buffer, offset, count, asyncResult=> { try { resultPort.Post(stream.EndRead(asyncResult)); } catch (Exception e) { resultPort.Post(e); } }, null); return resultPort; } Post result or exception

23. CopyStream static IEnumerator<ITask> CopyStream(Stream source, Stream dest) { try { varbuffer = new byte[blocksize]; intread = 0; do { Exception exception = null; yield return Arbiter.Choice( StreamAdapter.Read(source, buffer, 0, buffer.Length), r => { read = r; }, e => { exception = e; } ); if (exception == null) { // write to dest Choice arbiter: executes only one of its branches depending on the received message Port set on which to receive message Handler if int is received Handler if Exception is received

24. CopyStream static IEnumerator<ITask> CopyStream(Stream source, Stream dest) { try { varbuffer = new byte[blocksize]; intread = 0; do { varreadResult = StreamAdapter.Read( source, buffer, 0, buffer.Length); yield return (Choice)readResult; varexception = (Exception)readResult; if (exception == null) { // write to dest read = (int)readResult; Simplified notation

25. CopyStream • See the complete sample in Visual Studio

26. CopyStream II • Read and write are sequential • Modify to exploit parallelism • Read the next block while writing • Have to coordinate • “wait” until read succeeds and write succeedsor read failsor write fails Join Receivers Choice

27. CopyStream II while (write > 0) { varwriteResult = StreamAdapter.Write(dest, bufferB, 0, write); if (read > 0) { // read new bytes and write existing buffer readResult= StreamAdapter.Read(source, …); yield return Arbiter.Choice( Arbiter.JoinedReceive<int, EmptyValue>(false, readResult, writeResult, (r, s) => { read = r; } ), Arbiter.Receive<Exception>(false, readResult, e => { exception = e; }), Arbiter.Receive<Exception>(false, writeResult, e => { exception = e; }) ); Join branch: receives int on readResult and EmptyValue on writeResult Choice arbiter: only one of its branches will execute Join handler delegate gets both messages are parameters Receiver branches for exceptions

28. CopyStream II • See the complete sample in Visual Studio

29. User Interfaces • Concurrency desirable for responsivenes • UI often single threaded or has thread affinity • WinForms, WPF • Manual thread management, cross-thread calls • Use CCR to decouple UI from application logic • Adapter for handling cross thread calls  messages

30. User Interface - Adapters Application Logic (CCR Tasks) UI sends message to trigger application logic CCR to UI adapter (WPF or WinForms) CCR adapter handles message in UI context Application logic sends message to indicate completion or status

31. Exception Handling • Concurrent, async: try/catch wont do the job • Method 1: explicit handling • Exception caught at origin and posted as message for coordination • See CopyStream sample • Method 2: Causalities • Captures all exception thrown by tasks that are executed as result of the same cause (transitive) •  Nested exception handling in concurrent, asynchronous programs!

32. Causalities Create a new causality and add it to the active causalities of the current task Send a message and activate a receiver. All active causalities travel with the message to the tasks that handle the message. void ExecuteWithCausality() { varcausality = new Causality("my causality"); Dispatcher.AddCausality(causality); varport = new Port<int>(); port.Post(42); Arbiter.Activate(queue, Arbiter.Receive(false, port, HandleMessage) ); Arbiter.Activate(queue, Arbiter.Receive(false, (Port<Exception>)causality.ExceptionPort, Console.WriteLine ) ); } void HandleMessage(int message) { throw new Exception("Catch me if you can!"); } Handle the exception thrown within the causality.

33. DispatcherQueue • Manage queue of tasks • Tasks are picked round-robin from queues in scheduler • Parameters • Execution policy, queue length, scheduling rate • Use separate queues to • Separate tasks with significantly different arrival rates • Enforce scheduling policies

34. Dispatchers • Manage a thread pool • Default: 1 per core, or 2 on single core • Parameters • Number of threads, STA/MTA, Priority, Background • Use separate dispatchers • Interop with legacy components (COM) • Isolate “uncooperative” components • Separate tasks with significantly different lengths

35. Summary • Asynchronous message passing • Easy coordination of concurrent, asynchronous task • Compositional Arbiters • Sequential control flow with iterators • Exploits parallelism • Easy integration into .NET applications

36. Where to get it? • Microsoft CCR and DSS Toolkit 2008 • http://microsoft.com/ccrdss • Microsoft Robotics Developer Studio 2008 • http://microsoft.com/robotics • Express Version (free for non-commercial use) • Standard Version (free for academic institutions) • Internal on \\products

37. CCR based Components • Derive from CcrServiceBase • Stores reference to dispatcher queue • Overloads for Activate, TimeoutPort, etc • Model operations as messages • PortSet<SuccessResultType, Exception> for response • Expose operations port • Activate persistent receivers for operation messages

38. CCR based Components (cont) • Protect state with Interleave arbiter • TearDownReceiverGroup • Terminates Interleave • ExclusiveReceiverGroup • Message handlers that modify state • ConcurrentReceiverGroup • Message handlers that read state • Efficient reader/writer lock (writer biased) • Interleave protection spans iterative tasks!

39. CCR based Components (cont) public CcrComponent(DispatcherQueue queue) : base(queue) { Activate( Arbiter.Interleave( new TeardownReceiverGroup( Arbiter.Receive<StopOperation>(false, _operationsPort, StopHandler)), new ExclusiveReceiverGroup( Arbiter.ReceiveWithIterator<WriteOperation>(true, _operationsPort, WriteHandler)), new ConcurrentReceiverGroup( Arbiter.Receive<ReadOperation>(true, _operationsPort, ReadHandler)) ) ); }

40. Flowing Context • Culture, user principal, … • Explicit • Make it part of your message • Dispatcher • Useful if set of context is limited • E.g. one dispatcher per culture • Custom receivers • Capture context on post • Set it on execute

41. Performance • Intel Xeon 5150 (2x 2.66 GHz), 4 GByte • Persisted signal item receiver latency • Send message  schedule task  run task • 1.67 µs • Iterator latency • Spawn  schedule  run task • 2.1 µs