Too Slow?: Use VS2010 Profiling & Load Testing to Manage Performance Issues

1. Too Slow?: Use VS2010 Profiling & Load Testing to Manage Performance Issues Benjamin Day

2. Benjamin Day • Consultant, Coach, Trainer • Scrum.org Classes • Professional Scrum Developer (PSD) • Professional Scrum Foundations (PSF) • TechEd, VSLive, DevTeach, O’Reilly OSCON • Visual Studio Magazine, Redmond Developer News • Microsoft MVP for Visual Studio ALM • Team Foundation Server, TDD, Testing Best Practices,Silverlight, Windows Azure • http://blog.benday.com • benday@benday.com

3. Thanks. • To my content reviewers • Jerri Chiu, Microsoft • Dennis Bass, Microsoft

4. The Agenda • Basics of • Web Performance Tests • Load Tests • Load Test Rigs • Performance Explorer • Existing application • Identify performance problems • Go from bad performance to good • ASP.NET & WCF

5. Disclaimer: I’m going to use the term “load testing” imprecisely…a lot.

6. Here we go.

7. The Purpose of Load Testing • Identify the capabilities of your application • Possible goals • Explore • Verify • Find the limits • Crush

8. Best Practice:The Obligatory “Eat Your Vegetables” • Load test throughout the development cycle. • Catch problems early • Establish performance baseline • Watch for performance trends • Cheaper to fix early • Plan for performance

9. Cheaper To Fix Early

10. Dose Of Reality • You know you should work that way. • You probably don’t. • This talk is for you. • You have problems. • How do you figure out what’s going wrong?

11. Inspiration for this talk • Real customer engagement • Spent years developing a web application • Released it • Seriously grumpy customers • Huge hardware • ~20 simultaneous users • Fixed it • 4000+ simultaneous users • Reduced hardware

12. The Tools

13. demo Tour Of The Application

14. Disaster Strikes. • The application is deployed to production. • Users are losing their minds. • Intermittent errors on the site. • Site is slow. • Your boss is seriously angry. • “Fix it!”

15. The Problem. • “No Repro” • “Works on my box.” • You don’t know what the errors are. • The site seems to work fine. • Wild guess: it dies under load. • You need to create some load.

16. The Plan. • Script user activity with Web Performance Tests • Simulate a mix of users using Load Tests • Simulate lots of users with a Load Test Rig • Reproduce the errors...hopefully • Find code problems with the Performance Wizard • Fix the code

17. Web Performance Tests.

18. The Plan. • Script user activity with Web Performance Tests • Simulate a mix of users using Load Tests • Simulate lots of users with a Load Test Rig • Reproduce the errors...hopefully • Find code problems with the Performance Wizard • Fix the code

19. What is a Web Performance Test? • Test type in Visual Studio 2010 Ultimate • Simulate a user using an ASP.NET application • HTTP traffic • Building block of Load Tests

20. Tip: Web Tests & User Stories • Web Tests should simulate a User Story • Example: • As an administrator, I need to search for a person by username so that I can deactivate that person’s account. • As a customer, I want to order a pizza and pay with a credit card. • Helps to organize your suite

21. demo Create a basic Web Test

22. Data-driven Web Tests • Run web test once per row in data source • Simulate multiple users doing *similar* actions • Data source types • OLE DB, CSV, XML • Parameterization • Better tests, better coverage

23. demo Web Tests in the Underperforming Application

24. Load Tests.

25. The Plan. • Script user activity with Web Performance Tests • Simulate a mix of users using Load Tests • Simulate lots of users with a Load Test Rig • Reproduce the errors...hopefully • Find code problems with the Performance Wizard • Fix the code

26. What are Load Tests? • Mix of Web Performance and Unit Tests • Simulates a mix of users doing a mix of things • Simulate • Network speeds • Different browsers • Varying user activity loads

27. Why would you put a Unit Test in a Load Test? • Helpful for testing WCF • Cumbersome to simulate WCF via Web Test • (You’d have to re-invent the wheel.) • Unit tests can re-use the WCF libraries • (Done and done.) • Important for Silverlight and SOA • http://wcfloadtest.codeplex.com/ • http://sqlloadtest.codeplex.com/

28. (…and then you all say, “Hooray!”) demo Hey Kids, Let’s create some Load Tests!

29. Running Load Tests.

30. The Plan. • Script user activity with Web Performance Tests • Simulate a mix of users using Load Tests • Simulate lots of users with a Load Test Rig • Reproduce the errors...hopefully • Find code problems with the Performance Wizard • Fix the code

31. Two ways to run load tests. • Generate load from Visual Studio 2010 • Uses only 1 core/processor • Limited to 250 simulated users • Generate load from a Test Rig • Far better test • Simulate lots more users

32. Load Test Agent Licensing • Microsoft Visual Studio Team System 2010 Test Load Virtual User Pack • Simulate up to 1000 virtual users • Used to cost $$$

33. Visual Studio 2010 Load Test Feature Pack • Used to cost ~$3500.00 per 1000 simulated users • Free starting March 8, 2011 • Visual Studio 2010 Ultimate with MSDN • Simulate unlimited amount of load • (Limited only by your hardware.) • http://www.microsoft.com/visualstudio/en-us/products/2010-editions/load-test-virtual-user-pack

34. Load Test Rig & The Application • Controller • Orchestrates • Licensing • Perf Mon Counters • Agent (1..n) • Simulates users

35. The Plan. • Script user activity with Web Performance Tests • Simulate a mix of users using Load Tests • Simulate lots of users with a Load Test Rig • Reproduce the errors...hopefully • Find code problems with the Performance Wizard • Fix the code

36. demo Run Load Tests on the Rig

37. The Core Problem.

38. ‘sp_who2’ says lots of open connections.

39. Performance Profiling

40. The Plan. • Script user activity with Web Performance Tests • Simulate a mix of users using Load Tests • Simulate lots of users with a Load Test Rig • Reproduce the errors...hopefully • Find code problems with the Performance Wizard • Fix the code

41. Profiling Tools • Load Tests help you find issues in a system • “Surface” issues • Profiling helps you find issues in the code • Unit Tests, Web Tests, Load Tests • Repeatable • Good for targeted tuning • System under “natural” load • Not repeatable • Looking for rough patterns

42. Profiling Methods

43. Load Test Performance Sessions Via Wizard • Must be only Web Performance Tests • Can’t profile via Load Test Rig • ASP.NET must be running in x86 mode

44. demo Let’s go profile some stuff.

45. The Plan. • Script user activity with Web Performance Tests • Simulate a mix of users using Load Tests • Simulate lots of users with a Load Test Rig • Reproduce the errors...hopefully • Find code problems with the Performance Wizard • Fix the code

46. Any last questions?

47. For More Information: Articles http://tinyurl.com/3xjqgvz http://tinyurl.com/2ulvvvr

48. For More Information: Microsoft Links • Visual Studio Performance Testing Quick Reference Guidehttp://vsptqrg.codeplex.com/ • Content Index for Web Tests & Load Testshttp://tinyurl.com/ye97e52

49. For More Information: Microsoft Bloggers • VS Team Test Blog Site http://blogs.msdn.com/b/vstsqualitytools/ • Ed Glas - http://blogs.msdn.com/edglas/ • Bill Barnett - http://blogs.msdn.com/billbar/ • Sean Lumley - http://blogs.msdn.com/slumley/ • Dennis Stone - http://blogs.msdn.com/densto/ • Mike Taute- http://blogs.msdn.com/mtaute/ • NeeleshKamkolkar- http://blogs.msdn.com/b/nkamkolkar/

50. Thank you. blog.benday.com | www.benday.com | benday@benday.com