Maximize Your .NET Application Performance Task Parallel Library - TPL

1. Maximize Your .NET Application Performance Task Parallel Library - TPL 25th September 2014 by Romulus Susanu

2. Topics • Parallel Processing and Concurrency • Synchronous vs Asynchronous • Asynchronous Programming Patterns in .NET • Task Parallel Library - TPL • Data Parallelism • Task Parallelism

3. Parallel Processing and Concurrency • keep your application responsive • maximize the performance of your code

4. Parallel Processing vs Concurrency • Parallelism • refers to techniques to make programs faster by performing several computation in parallel. • requires hardware with multiple CPU • Key problem • reduce data dependencies

5. Parallel Processing vs Concurrency • Concurrency • refers to techniques that make program more usable • can be implemented and is used a lot on single CPU • multi-tasking operating system is synonym for supporting concurrency

6. Parallel Processing vs Concurrency

7. Synchronous vs Asynchronous • in .NET it's directly related with threads • in a threaded system the decision to suspend one thread and execute another • is largely outside of the programmer’s control • threaded model execution are handled by the OS

8. Synchronous vs Asynchronous • Synchronous • wait for it to finish before moving on to another task • Asynchronous • move on to another task before it finished • executing a process/task on another thread • Why asynchronous programming is more performing?

9. Synchronous vs Asynchronous Why asynchronous programming is more performing? Blocking in a synchronous program The asynchronous model • Why would a task be blocked?

10. Synchronous vs Asynchronous • Why would a task be blocked? • waiting to perform I/O • to transfer data to or from an external device. • An asynchronous program is often called a non-blocking program!

11. Synchronous vs Asynchronous • When the asynchronous model performs best? • there are a large number of tasks so there is likely always at least one task that can make progress. • the tasks perform lots of I/O, causing a synchronous program to waste lots of time blocking when other tasks could be running. • a network server implementation is a prime candidate for the asynchronous model • Every time the OS transfers control over from one thread to another it has to save all the relevant registers, memory map, stack pointers, FPU context etc. so that the other thread can resume execution where it left off.

12. Asynchronous Programming Patterns in .NET • Asynchronous Programming Model (APM) (also called IAsyncResult Pattern) • Event-based Asynchronous Pattern (EAP) • Task-based Asynchronous Pattern (TAP)

13. Asynchronous Programming Patterns in .NET • Asynchronous Programming Model (APM) • asynchronous operations require Begin and End methods • no longer recommended for new development

14. Asynchronous Programming Patterns in .NET • Asynchronous Programming Model (APM) • Code sample: • consider a Read method that reads a specified amount of data into a provided buffer starting at a specified offset • publicclassMyClass • { • publicIAsyncResultBeginRead(byte [] buffer, int offset, int count, • AsyncCallback callback, object state); • publicintEndRead(IAsyncResultasyncResult); • }

15. Asynchronous Programming Patterns in .NET • Event-based Asynchronous Pattern (EAP) • requires a method that has the Async suffix and one or more events • was introduced in the .NET Framework 2.0 • no longer recommended for new development

16. Asynchronous Programming Patterns in .NET • Event-based Asynchronous Pattern (EAP) • Code sample: • publicclassAsyncExample • { • // Synchronous methods. • publicint Method1(stringparam); • // Asynchronous methods. • publicvoid Method1Async(stringparam); • publicevent Method1CompletedEventHandler Method1Completed; • publicvoidCancelAsync(); • publicboolIsBusy { get; } • } • BackgroundWorker components represent more complex implementations of EAP (RunWorkerAsync(), CancelAsync, ProgressChanged, RunWorkerCompletedetc.)

17. Asynchronous Programming Patterns in .NET • Task-based Asynchronous Pattern (TAP) • uses a single method to represent the initiation and completion of an asynchronous operation • was introduced in the .NET Framework 4.0 • recommended approach to asynchronous programming

18. Asynchronous Programming Patterns in .NET • Task-based Asynchronous Pattern (TAP) • Code sample: • publicclassMyClass • { • public Task<int> ReadAsync(byte [] buffer, int offset, int count); • } • System.Threading.Tasks Namespace • provides types that simplify the work of writing concurrent and asynchronous code

19. Task Parallel Library - TPL • Data Parallelism • Task Parallelism

20. Task Parallel Library - TPL • Data Parallelism • refers to scenarios in which the same operation is performed concurrently • the source collection is partitioned so that multiple threads can operate on different segments concurrently • data parallelism through the System.Threading.Tasks.Parallel class • Parallel class provides method-based parallel implementations of for and foreach loops

21. Task Parallel Library - TPL • Data Parallelism • The SumRootN method returns the sum of the nth root of all integers from one to 10 million, where n is a parameter

22. Task Parallel Library - TPL • Data Parallelism

23. Task Parallel Library - TPL • Task Parallelism • a task represents an asynchronous operation • in some ways it resembles the creation of a new thread or ThreadPool work item • // Define and run the task. • Task taskA = Task.Run( () => Console.WriteLine("Hello from taskA."));

24. Task Parallel Library - TPL • Task Parallelism • Tasks provide two primary benefits: • more efficient and more scalable use of system resources • lock-free algorithm - ConcurrentQueue<T> • Hill Climbing algorithm • Work-Stealing algorithm • more programmatic control than is possible with a thread or ThreadPool work item

25. Task Parallel Library - TPL • Hill Climbing algorithm • to improve the utilization of cores when threads are blocked by I/O or other wait conditions that stall the processor • the .NET thread pool has an opportunity to inject threads every time a work item completes or at 500 millisecond intervals, whichever is shorter • if adding threads seems to be helping throughput, the thread pool adds more; otherwise, it reduces the number of worker threads.

26. Task Parallel Library - TPL • Work-stealing algorithms • ThreadPool Global Queue vs. Local Queues • Extra efficiencies: • improved cache locality • minimized contention

27. Task Parallel Library - TPL • Work-stealing algorithms • What happens when a thread's local work queue is empty and the global queue is also empty? • Benefits: • Load-balancing

28. Task Parallel Library - TPL • TPL - More programmatic control • Creating and Running Tasks Implicitly • just pass in an Action delegate for each item of work • Parallel.Invoke(() => DoSomeWork(), () => DoSomeOtherWork());

29. Task Parallel Library – TPL TPL - More programmatic control • Creating and Running Tasks Explicitly • a task that does not return a value is represented by the System.Threading.Tasks.Task class • the Task.Wait method to ensure that the task completes execution before the console mode application ends Thread.CurrentThread.Name = "Main"; // Create a task and supply a user delegate by using a lambda expression. Task taskA = new Task( () => Console.WriteLine("Hello from taskA.")); // Start the task. taskA.Start(); // Output a message from the calling thread. Console.WriteLine("Hello from thread '{0}'.", Thread.CurrentThread.Name); taskA.Wait(); // The example displays the following output:  // Hello from thread 'Main'.  // Hello from taskA.

30. Task Parallel Library – TPL TPL - More programmatic control • Creating and Running Tasks Explicitly • Task.Run method • the Run methods use the default task scheduler • The Run methods are the preferred way to create and start tasks when more control over the creation and scheduling of the task is not needed Thread.CurrentThread.Name = "Main"; // Define and run the task. Task taskA = Task.Run( () => Console.WriteLine("Hello from taskA.")); // Output a message from the calling thread. Console.WriteLine("Hello from thread '{0}'.", Thread.CurrentThread.Name); taskA.Wait(); // The example displays the following output:  // Hello from thread 'Main'.  // Hello from taskA.

31. Task Parallel Library – TPL TPL - More programmatic control • Creating and Running Tasks Explicitly • TaskFactory.StartNewmethod • Use this method when: • creation and scheduling do not have to be separated • require additional task creation options • use of a specific scheduler Thread.CurrentThread.Name = "Main"; // Better: Create and start the task in one operation. Task taskA = Task.Factory.StartNew(() => Console.WriteLine("Hello from taskA.")); // Output a message from the calling thread. Console.WriteLine("Hello from thread '{0}'.", Thread.CurrentThread.Name); taskA.Wait(); // The example displays the following output:  // Hello from thread 'Main'.  // Hello from taskA.

32. Task Parallel Library – TPL TPL - More programmatic control Creating and Running Tasks Explicitly System.Threading.Tasks.Task<TResult> class Task<Double>[] taskArray = { Task<Double>.Factory.StartNew(() => DoComputation(1.0)), Task<Double>.Factory.StartNew(() => DoComputation(100.0)), Task<Double>.Factory.StartNew(() => DoComputation(1000.0)) }; var results = new Double[taskArray.Length]; Double sum = 0; for (inti = 0; i < taskArray.Length; i++) { results[i] = taskArray[i].Result; Console.Write("{0:N1} {1}", results[i], i == taskArray.Length - 1 ? "= " : "+ "); sum += results[i]; } Console.WriteLine("{0:N1}", sum); } // The example displays the following output:  // 606.0 + 10,605.0 + 100,495.0 = 111,706.0 privatestatic Double DoComputation(Double start) { Double sum = 0; for (var value = start; value <= start + 10; value += .1) sum += value; return sum; }

33. Task Parallel Library – TPL TPL - More programmatic control • Continuation Tasks • in asynchronous programming, it is very common for one asynchronous operation, on completion, to invoke a second operation and pass data to it // The antecedent task. Can also be created with Task.Factory.StartNew. Task<DayOfWeek> taskA = new Task<DayOfWeek>(() => DateTime.Today.DayOfWeek); // The continuation. Its delegate takes the antecedent task  // as an argument and can return a different type. Task<string> continuation = taskA.ContinueWith( (antecedent) => { returnString.Format("Today is {0}.", antecedent.Result); }); // Start the antecedent. taskA.Start(); // Use the contuation's result. Console.WriteLine(continuation.Result);

34. Task Parallel Library – TPL TPL - More programmatic control • Task Creation Options var task3 = new Task(() => MyLongRunningMethod(), TaskCreationOptions.LongRunning); task3.Start();

35. Task Parallel Library – TPL TPL - More programmatic control • Continuation Tasks • in asynchronous programming, it is very common for one asynchronous operation, on completion, to invoke a second operation and pass data to it // The antecedent task. Can also be created with Task.Factory.StartNew. Task<DayOfWeek> taskA = new Task<DayOfWeek>(() => DateTime.Today.DayOfWeek); // The continuation. Its delegate takes the antecedent task  // as an argument and can return a different type. Task<string> continuation = taskA.ContinueWith( (antecedent) => { returnString.Format("Today is {0}.", antecedent.Result); }); // Start the antecedent. taskA.Start(); // Use the contuation's result. Console.WriteLine(continuation.Result);

36. Task Parallel Library – TPL TPL - More programmatic control • Task Schedulers • makes sure that the work of a task is eventually executed • the default task scheduler is based on the .NET Framework 4 ThreadPool • Custom Task Schedulers • ParallelExtensionsExtras package • LimitedConcurrencyLevelTaskScheduler - limits the number of threads used by the application • OrderedTaskScheduler - guarantees that tasks are processed in the order that they were scheduled. • ThreadPerTaskScheduler - dedicates a thread to each individual task

37. Task Parallel Library – TPL TPL - More programmatic control • Real Case • foreach (SurveyDetailsurveyDetailinsurveyDetailList) • {                  soFar++; SurveyProcessing.PopulateUPSInformation(surveyDetail); } webconfig privatestaticreadonlyLimitedConcurrencyLevelTaskScheduler _lcts =  newLimitedConcurrencyLevelTaskScheduler(50); … IList<Task> tasks = newList<Task>();              vartaskFactory = newTaskFactory(_lcts);              foreach (SurveyDetailsurveyDetailinsurveyDetailList)              {                 varverifyByUpsTask = taskFactory.StartNew( • () => PopulateUPSInformation(surveyDetail)); }          <connectionManagement> <addaddress="*"maxconnection="48" /> </connectionManagement>

38. Thank you!