An Insider’s View to Concurrency at Microsoft

An Insider’s View to Concurrency at Microsoft Stephen Toub (stoub@microsoft.com) Parallel Computing Platform Microsoft Corporation

Agenda • Why We (I) Care • What We’ve Built for Developers • What We’re Building for Developers

Moore’s Law: Alive and Well http://upload.wikimedia.org/wikipedia/commons/2/25/Transistor_Count_and_Moore%27s_Law_-_2008_1024.png

MS Apps Using ParallelismExample: Visual Studio • Background compilation • Regex-based file search • Graph layout • Reference highlighting • IntelliSense sorting • Project build system • Code analysis • Unit testing • …

But it’s not just about core count… • Increasingly connected applications • More latency • e.g. everything as a service • More UI responsiveness problems • e.g. the toilet bowl of death • More scalability issues • Server, Cloud • e.g. streaming data sources, data distribution • Client • e.g. modeling interacting biological entities • Async-only APIs • e.g. Silverlight

The Challenges of Concurrency • A different way of thinking • Forcing the square peg (concurrency) into the round hole (sequential) is a common (inadequate) mistake • Parallel patterns are not prevalent, well known, nor easy to implement • CS curriculum is still largely “sequential”: Cormen, et. al; Knuth; etc. • A different way of writing software • Indeterminate program behavior • No longer: “A happens, then B happens, then C happens” • It’s just: “A, B, and C happen” • With distinct costs, too • Deadlocks, livelocks, latent race conditions, priority inversions, hardware-specific memory model reordering, … • Businesses have little desire to go deep • Best devs should focus on business value, not concurrency • Need simple ways to allow all devs to write concurrent code

Example: Searching and Sorting IEnumerable<RaceCarDriver> drivers = ...; varresults = new List<RaceCarDriver>(); foreach(var driver in drivers) { if (driver.Name == queryName && driver.Wins.Count >= queryWinCount) { results.Add(driver); } } results.Sort((b1, b2) => b1.Age.CompareTo(b2.Age));

Manual Parallel Solution IEnumerable<RaceCarDriver> drivers = …; varresults = new List<RaceCarDriver>(); intpartitionsCount = Environment.ProcessorCount; intremainingCount = partitionsCount; varenumerator = drivers.GetEnumerator(); try { using (vardone = new ManualResetEvent(false)) { for(inti = 0; i < partitionsCount; i++) { ThreadPool.QueueUserWorkItem(delegate { while(true) { RaceCarDriver driver; lock (enumerator) { if (!enumerator.MoveNext()) break; driver = enumerator.Current; } if (driver.Name == queryName && driver.Wins.Count >= queryWinCount) { lock(results) results.Add(driver); } } if (Interlocked.Decrement(refremainingCount) == 0) done.Set(); }); } done.WaitOne(); results.Sort((b1, b2) => b1.Age.CompareTo(b2.Age)); } } finally { if (enumerator is IDisposable) ((IDisposable)enumerator).Dispose(); }

P LINQ Solution IEnumerable<RaceCarDriver> drivers = ...; varresults = from driver in drivers where driver.Name == queryName && driver.Wins.Count >= queryWinCount orderbydriver.Ageascending select driver; .AsParallel()

Demo PLINQ

Visual Studio 2010Tools, Programming Models, Runtimes Tools Programming Models Visual Studio 2010 .NET Framework 4 Visual C++ 2010 Parallel LINQ Parallel Debugger Tool Windows Parallel Pattern Library Async AgentsLibrary Task Parallel Library Data Structures Data Structures Concurrency Runtime • Profiler • Concurrency Visualizer Task Scheduler ThreadPool Task Scheduler Resource Manager Resource Manager Operating System Windows Threads UMS Threads Key: Managed Native Tooling

Investigations for the future…

This is somesynchronous code with .NET 4… public voidCopyStreamToStream(Stream source, Stream destination){byte[] buffer = newbyte[0x1000];intnumRead;while ((numRead= source.Read(buffer, 0, buffer.Length)) != 0) {destination.Write(buffer, 0, numRead);}}

publicIAsyncResultBeginCopyStreamToStream( Stream source, Stream destination){vartcs = newTaskCompletionSource<object>();byte[] buffer = newbyte[0x1000]; Action<IAsyncResult> readWriteLoop = null;readWriteLoop = iar => {try {for (boolisRead = iar == null; ; isRead = !isRead) {switch (isRead) {casetrue:iar = source.BeginRead(buffer, 0, buffer.Length, readResult => {if (readResult.CompletedSynchronously) return;readWriteLoop(readResult); }, null);if (!iar.CompletedSynchronously) return;break;casefalse:intnumRead = source.EndRead(iar);if (numRead == 0) {tcs.TrySetResult(null);return; }iar = destination.BeginWrite(buffer, 0, numRead, writeResult => {if (writeResult.CompletedSynchronously) return;destination.EndWrite(writeResult);readWriteLoop(null); }, null);if (!iar.CompletedSynchronously) return;destination.EndWrite(iar);break; } } }catch (Exception e) { tcs.TrySetException(e); } };readWriteLoop(null); returntcs.Task;} publicvoidEndCopyStreamToStream(IAsyncResultasyncResult){ ((Task)asyncResult).Wait();} This is an expert’s asynchronous code with .NET 4… public voidCopyStreamToStream(Stream source, Stream destination){byte[] buffer = newbyte[0x1000];intnumRead;while ((numRead= source.Read(buffer, 0, buffer.Length)) != 0) {destination.Write(buffer, 0, numRead);}}

A compiler could do the work for us… public voidCopyStreamToStream(Stream source, Stream destination){byte[] buffer = newbyte[0x1000];intnumRead;while ((numRead= source.Read(buffer, 0, buffer.Length)) != 0) {destination.Write(buffer, 0, numRead);}} public TaskCopyStreamToStream(Stream source, Stream destination){byte[] buffer = newbyte[0x1000];intnumRead;while ((numRead= await source.ReadAsync(buffer, 0, buffer.Length)) != 0) { await destination.WriteAsync(buffer, 0, numRead);}} public TaskCopyStreamToStream(Stream source, Stream destination){byte[] buffer = newbyte[0x1000];intnumRead;while ((numRead= awaitsource.ReadAsync(buffer, 0, buffer.Length)) != 0) {await destination.WriteAsync(buffer, 0, numRead);}}

Q&A

An Insider’s View to Concurrency at Microsoft