Load Testing Approach

1. Load Testing Approach Quick Guide to Plan and Execute Load Test Using Load Runner! -Ranjit jcrvs@hotmail.com

2. Why Is a Approach Needed? • Quality approach work smarter to be faster. • Do it right first time, every time. • Detailed approach that prevent from getting lost in the “load testing sea”. • Use of smart tools load runner.

3. Conventional Approach • Random requirements from management fro load test • In-experienced staff • No requirements analysis performed for load test • Easy to get carried away in load testing

4. The Approach • Perform requirement analysis to get information as • Type of load test • User load estimation • User load distribution • User activity analysis • Production environment analysis • Database size • Report generation • Simple metric collection

5. Type of Load Test • Determine the goal of the test • Do we intent to find the synchronization problems? • Do we intend to perform a load test? Or a stress test? Or a performance test?

6. Synchronization Problem Tracing Load Test • Goal of this test is to determine the synchronous issues and trouble some areas. • This test may need vu-generated scripts with lots of rendezvous points or rendezvous points at each post/action.

7. Load/stress/performance • Different type of tests : • Load test determines how is the performance of application under the concurrent user sessions for typical user scenario. The think time taken into consideration in these test scripts. • Stress test examines how application behaves under maximum load. In simple terms find the upper threshold for the application below which it can work normally. Think time ignored in these tests. • Performance test indicates response time for the entire application from the user’s perspective.

8. What Do U Want? • Do determine what kind of test do you want. • Then plan ahead.

9. User Load Estimation • A detailed feed back form the marketing/business development will give a idea of the user load or the number of users using the product. This will determine the load to be used against the product in testing. • An inexperienced staff may configure a load test to simulate 1000 v-user but user base for the application may be not more than 400. This may result in licenses being lost for the v-user and time and efforts as well.

10. User Load Distribution • A detailed feed back form the marketing/business development will give a idea of the user load distribution. • The user load may be peak in the morning or afternoon and very less in the evening. • This may also help determine the concurrent users and simultaneous users.

11. User Load Distribution • This factor will input to the scenarios to be used and configured in the load test. • E.G. BD may come up with a user load of 1000 users per day and each grouped to perform certain activity as say 30% in section A of application rest in section B of the application etc.

12. User Activity Analysis • A detailed discussion with marketing/BD may reveal the user activity details on which detailed scripts can be written. • E.G. In morning there may be 200 forms submitted or in evening most of the users may login and perform results generation activity so in morning scenario out of 1000 concurrent user, 70% may be performing this activity.

13. Production Environment Analysis • A discussion with the I.T. Department will throw light on deployment environment. • This may input valuable information as does the environment have enough hardware? Is it running in a clustered environment?

14. Production Environment Analysis • Mirror the production environment into a test bed. • Alternatively create test bed and after load test suggest production environment to management.

15. Database Size • The database size does matter in the load test. • Bulky the database, more the latency. • Define the database size prior to load test e.G. Conducting first set of test on a 50K sized DB then a 100K sized DB and so forth.

16. Report Generation • Generate two types of reports • Load runner detailed reports for engineering department • Generic reports for the activities performed in the scenario and response time details with other observation and conclusions for the management • Generating graphs also helps • Helps in tracking per load cycle results • Easy interpretation for people

17. Simple Metric Collection • Basic metrics can be collected as follows: • CPU utilization : should not exceed 60%. • Per page size. This is a indicator of the bulky-ness of the page. Response time for each transaction. • Configuration details. • Test bed details.

18. Simple Metric Collection • Total hits and hits/second, should not be greater than 20 or request queue details need to be collected. • % Failed transactions, should not be greater than 5%. • Number of processes running on server. • Memory details.

19. Simple Metric Collection • Load runner can be used to collect the above metrics. • Other option is to use O.S. Specific tools.

20. Plan on the Above Analysis – Clubbing All Together • After the requirement analysis, feed all the data into a test plan. • Test plan will contain the type of test to perform, test bed details, load/user details and activity scenarios.

21. Plan on Scenarios • Plan scenarios for execution based on the above input and different daytime for the day as morning scenario, afternoon scenario etc. • Each scenario may scale to 3 to 4 hours. So activity analysis as number of forms submitted per day etc and time may help to set the number of iteration for each scenario.

22. Follow up Action:scripts • Create test scripts with vu-generator for the interactive activities to be performed with application. • Unit test in vu-generator debug mode. • Also run the same in 10 user 10 iteration load runner scenario mode. Use the option “show v-user” to check for the activities being performed. • Parameterize scripts and co-related them. • Make scripts dynamic, more efforts to be put in for making dynamic than just co-relating the scripts.

23. More on Scripts • Add comments and queries to fetch the parameterized data from database to make it user independent. • Logs may reveal error which at time may be hidden. • In runtime settings, use option as mark each step as transaction to prevent adding manually transaction function for each step.

24. Configuring Scenarios • Recommended to unique create groups, each group is represented by one script. • Use ramp-up and ramp-down features to prevent instantaneous loads on load test server. Load runner gives one click access to pre-configured “slow ramp-up” and “ramp up” scenario. • If scripts are dynamic then recommended to use duration based scenario. • More refined control can be achieved for scenario by scheduling by group rather than scheduling by scenario.

25. Configuring Scenarios • Enable standard logging. • One click generator configuration can be performed using load runner. • In runtime settings, use option as mark each step as transaction to prevent adding manually transaction function for each step. • The think time in-between two iterations is recommended in load tests. • Proxy may or may not be needed. Recommended not to use proxy to get a benchmark results first, then run the same scripts though proxy and compare results.

26. Configuring Scenarios • Simulating browser cache option may be turned off to make sure each request is fetched form the server. • Bandwidth throttling may or may not be used as per requirements. • Enable web performance graphs option in load runner to monitor at test runtime the results.

27. Configuring Scenarios • Disable or enable “continue on error” option. Recommended to enable it for cluster fail-over tests and also regular load test. • The option to run v-user as a process or thread may be used as per requirements. Recommended to use option is thread to prevent overhead of the memory and system resources. But also more threads per process makes the process unstable. Load runner allows configuration of number of thread per process for stability.

28. Execute Tests and Report • Prepare test bed and execute planned scenarios. • Recommended server restart before each test cycle. • Generate reports for each cycle.

29. In a Nutshell • Planned approach make life simple and makes load test simple too. • Requirement analysis is important activity before starting load test. • Metrics are the end results so save results for each cycle and analyze. • FLY HIGH with simple minds and simple practices !!

30. Glossary • LR- load runner • O.S. – Operating system • BD – business development

31. Q & A