Manycore and .NET 4: A Match Made in Visual Studio 2010

1. FT03 Manycore and .NET 4:A Match Made inVisual Studio 2010 Stephen Toub Parallel Computing Platform Microsoft Corporation

2. Parallel Computing and PDC09 • Overview • FT07:The State of Parallel Programming • WKSP08: Patterns of Parallel Programming • Managed code in Visual Studio 2010 • FT03:Manycore and .NET 4: A Match Made in Visual Studio 2010 • FT21:PLINQ: LINQ, but Faster! • FT20:F# for Parallel and Asynchronous Programming • Native code in Visual Studio 2010 • SVR18: Developing Applications for Scale-Up Servers Running Windows Server 2008 R2 • SVR10: Lighting up Windows Server 2008 R2 Using the ConcRT on UMS • FT19:C++ Forever: Interactive Applications in the Age of Manycore • HPC Server • SVR01:Accelerating Applications Using Windows HPC Server 2008 • Research and Incubation • VTL02:Axum: A .NET Language for Safe and Scalable Concurrency • VTL32:Concurrency Fuzzing & Data Races • SVR17:Data-Intensive Computing on Windows HPC Server with DryadLINQ • VTL04:Rx: Reactive Extensions for .NET • FT36:Future of Garbage Collection

3. Agenda • Why Parallelism, Why Now? • Task Is Your New Best Friend • Execution Efficiency • A Reference To Your Work • The Power of Coordination and Dataflow • A Building Block for the Future • Rev up your Loops • (PLINQ covered in FT21) • The Power of a One-Line Change • Visualize Parallel • A Picture is Worth A Thousand Words

4. Moore’s Law “The number of transistors incorporated in a chip will approximately double every 24 months.” Gordon Moore Intel Co-Founder http://www.intel.com/pressroom/kits/events/moores_law_40th/

5. The Manycore Shift • “[A]fter decades of single core processors, the high volume processor industry has gone from single to dual to quad-core in just the last two years. Moore’s Law scaling should easily let us hit the 80-core mark in mainstream processors within the next ten years and quite possibly even less.”-- Justin Rattner, VP, Intel (February 2007) • “If you haven’t done so already, now is the time to take a hard look at the design of your application, determine what operations are CPU-sensitive now or are likely to become so soon, and identify how those places could benefit from concurrency.”-- Herb Sutter, C++ Architect at Microsoft (March 2005)

6. Parallel Computing and PDC09 Tools Overviews: FT07 & WKSP08 Managed Languages VTL02 Visual F# FT20 Axum FT03 FT19 Visual Studio 2010 Parallel Debugger Windows Native Libraries Managed Libraries DryadLINQ SVR17 FT19 Async AgentsLibrary Parallel Pattern Library • Profiler Concurrency • Analysis FT21 Parallel LINQ VTL04 Rx FT03 Task ParallelLibrary Data Structures Data Structures VTL32 Microsoft Research Native Concurrency Runtime SVR10 Task Scheduler Race Detection Managed Concurrency Runtime Resource Manager ThreadPool Fuzzing Operating System SVR01 HPC Server Threads UMS Threads SVR18 Windows 7 / Server 2008 R2 Research / Incubation Visual Studio 2010 / .NET 4 Operating System Key:

7. Minimizing the Overhead of Parallelism • The Argument: • Parallelization demands division • Executing work items has overhead • Results in increased overhead to work ratio • We must decrease overhead Overhead Overhead Overhead Work Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead Overhead

8. ThreadPool in .NET 3.5 Global Queue Worker Thread 1 Worker Thread 1 … Item 4 Item 5 Program Thread Item 1 Item 2 Item 3 Item 6 • Thread Management: • Starvation Detection • Idle Thread Retirement

9. ThreadPool in .NET 4 Local Work-Stealing Queue Local Work-Stealing Queue Lock-Free Global Queue … Worker Thread 1 Worker Thread p … Task 6 Task 3 Program Thread Task 4 Task 1 Task 5 Task 2 • Thread Management: • Starvation Detection • Idle Thread Retirement • Hill-climbing

10. Task is Your New Best Friend • ThreadPool.QueueUserWorkItem • Great for fire-and-forget • But what about… • Waiting • Canceling • Continuing • Composing • Exceptions • Dataflow • Debugging • …

11. System.Threading.Tasks.Task +System.Collections.Concurrent demo

12. New Sync Primitives in .NET 4 • Public, and used throughout PLINQ and TPL • Address many of today’s core concurrency issues • Thread-safe, scalable collections • IProducerConsumerCollection<T> • ConcurrentQueue<T> • ConcurrentStack<T> • ConcurrentBag<T> • ConcurrentDictionary<TKey,TValue> • Phases and work exchange • Barrier • BlockingCollection<T> • CountdownEvent • Partitioning • {Orderable}Partitioner<T> • Partitioner.Create • Exception handling • AggregateException • Initialization • Lazy<T> • LazyInitializer.EnsureInitialized<T> • ThreadLocal<T> • Locks • ManualResetEventSlim • SemaphoreSlim • SpinLock • SpinWait • Cancellation • CancellationToken{Source}

13. Rev Up Your Loops • Control flow is a primary source of work • Parallelizable when iterations are (or can be made) independent • Synchronous • All work quiesces, regularly or exceptionally • Lots of knobs • Cancelation, breaking, task-local state, custom partitioning, scheduling, degree of parallelism • Visual Studio 2010 profiler support (as with PLINQ) foreach(var item in data) { work(item); } for (int i = 0; i < n; i++) { work(i); } StatementA(); StatementB; StatementC(); Parallel.ForEach(data, item=> { work(item); }); Parallel.For(0, n, i=> { work(i); }); Parallel.Invoke( () => StatementA(), () => StatementB, () => StatementC());

14. Efficiency and Load Balancing Static Scheduling (Range) DynamicScheduling CPU0 CPU1 … CPUN CPU0 CPU1 … CPUN 5 5 7 1 3 7 1 3 6 6 8 8 2 2 4 4

15. Fun With Loops demo

16. Visualize Parallel • Concurrency Profiler • Anti-Patterns You Can See • Parallel Debugger • Real-time insight into multithreaded behavior

17. Concurrency Profiler Views +Parallel Debugger Toolwindows demo

18. To Infinity And Beyond… • The “Manycore Shift” is happening • Parallelism in your code is inevitable • Visual Studio 2010 and .NET 4 will help • Key Links • Parallel Computing Dev Center • http://msdn.com/concurrency • Code samples • http://code.msdn.microsoft.com/ParExtSamples • Blogs • Managed: http://blogs.msdn.com/pfxteam • Tools: http://blogs.msdn.com/visualizeparallel • Forums • http://social.msdn.microsoft.com/Forums/en-US/category/parallelcomputing We love feedback! (and high session scores ;)

19. YOUR FEEDBACK IS IMPORTANT TO US! Please fill out session evaluation forms online at MicrosoftPDC.com

20. Learn More On Channel 9 • Expand your PDC experience through Channel 9 • Explore videos, hands-on labs, sample code and demos through the new Channel 9 training courses channel9.msdn.com/learn Built by Developers for Developers….