Rational Suite PerformanceStudio Tips & Techniques

1. Rational Suite PerformanceStudioTips & Techniques Kent Siefkes Rational Software

2. Introduction • Something for everyone • Beginner • Intermediate • Advanced • Survey says . . . • Thanks to the Tech Support staff, Dawn Haynes, and Jeff Robbins for their input

3. Agenda – Beginner & Intermediate Topics • Getting Started • Install • Uninstall • Setup • Recording • Going to Multi-User • Scheduling • Synchronization Points • Transactors

4. Agenda – Advanced Topics • Miscellaneous Advanced Tips • VU Performance Tuning • HTTP basic encoding and parameterization • HTTP response verification

5. Getting Started with Rational Suite PerformanceStudio • Tips in this section focus on the things likely to be encountered when first using PerformanceStudio • Level: Beginner

6. Install • Licensing • Rational Suite PerformanceStudio  Readme for Rational Suite PerformanceStudio (Please Read Me!) • New Features • Supported Playback Platforms • General Notes (tips & advice) • Known Limitations

7. Uninstall/Install: RSPS Network Driver • When uninstalling RSPS, be sure to uninstall the network driver! • NOTE: Always install the new network driver with each release of RSPS • On NT, in the Settings  Networkdialog, select the driver on the Services tab and click Remove • If you can’t find the RSPS driver in the Services tab ...

8. Uninstall: RSPS Network Driver • You can manually remove the RSPS network driver: • Using regedt32 - go to HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\ • Remove the NetworkPacket folder • Reboot

9. Setup: Configuration • Configuration Setup for Large User Counts • Reference source: Using Rational LoadTest Appendix B • NT Environment Variables • TEST7_LTMASTER_SHM_MINSZ • Limit on Master is for total user count • Limit on Agent is for that computer only • Formula: 700 x # of users (default good for 1000 users) • TEST7_LTMASTER_NTUSERLIMIT • Limit per specific Master or Agent • Formula: Equal to # of users per machine (default is 1000)

10. Setup: Nutcracker Settings Default Settings • NuTCRACKER Control Panel Applet • NuTC 4 Options  Semaphore Settings • Default settings are sufficient for about 245 users • Formula: 10 + # of users + # of shared variables

11. Setup: Repository • Repository Configuration Issues • Choose repository location with ample disk space • Use local disk whenever possible – otherwise users on the Master write log files across the network during test runs • Use UNC path names if you plan on using GUI Agents Repository

12. Setup: Miscellaneous • Check disk space in System Temp directory on Master and Agents • Redirect if necessary (using either [TMPDIR], TMP or TEMP environment variables in that order of precedence) to somewhere that has room • For Agents, you can use the Schedule’s Computer dialog to put temporary user log files, datapools and compiled scripts in a different location • Specify location with Local Directory field

13. Recording Methods • API Recording (default) • Records the application’s use of specific API’s: • OCI, ODBC, WinInet (IE), Netscape, WinSock • Network Recording (“on-the-wire”) • Uses special RSPS-provided special network drivers • Ethernet (default) or Token-Ring • Proxy Recording • Usually not needed (API or network should suffice) • Uses a special intermediary proxy that you set up between the client and server • Client or system administration is required to direct the application’s traffic to the proxy

14. HTTP API Recording Tip • Use the Program arguments in the VU Start Application dialog to specify the target URL • Internet Explorer or Netscape • API Recording only

15. API Recording Tip • Closing the application under test while recording can yield unexpected behavior! • Stop Recording popup dialog can get buried • Can look like Robot is hanging

16. Mental Break • Questions? • Anyone have a beginner tip to add? • We’re moving on to the next level … • You ready?

17. Going to Multi-User • This section focus on multi-user scheduling concepts and tips • Synchronization Points • Transactors • Level: Intermediate

18. Synchronization Point Basics • Each user waits until the last scheduled user arrives • # of users to sync is calculated automatically • Release Together or Staggered within an interval • Staggered release is random

19. Synchronization Point Tips • Schedule defensively! • Ensure that ALL users enter a scheduled Sync Point • Don’t insert a Sync Point in a Random Selector! • Abnormal terminations could cause active users to bottleneck at a Sync Point • Set the Timeout parameter, or ... • Control Sync Point behavior manually • Sync Point Monitor View is interactive • Right click on Sync Point and select Release from context menu -- illustration on next Page • Sync Point can be released at any time during the schedule run

20. Manually Releasing a Sync Point Click to get Sync Points View Right-click Menu

21. Running Users Concurrently with Sync Points • Releasing users “Together” will create a bottleneck • Recommended settings for running users concurrently • Release Together • Restart time • Set to 5 or 10 seconds

22. Staggering Users – One Method • Question: How do I release users at fixed intervals? • Solution: • Use Release Together (no Restart time) • Create a “delay” script and schedule it after the Sync Point • Script contents if ALL users participate in the Sync Point: delay(_uid * SECONDS(5)) /* replace 5 with desired interval */ • Script contents if a subset of users participate: shared myturn; delay(myturn++ * SECONDS(5)) /* replace 5 with desired interval */

23. Transactors • New to Rational Suite PerformanceStudio Version 2000.02.10 • Provides “Transaction Rate” pacing • Submission of next transaction is based on achieving a particular rate • Contrast to the think time pacing model for user modeled behavior to control transaction rate • “Rate” of work is “think time” + “server response time” • Think time is not shortened if the server is slow

24. Transactor Basics • A transaction is defined by a Scenario • Inter-arrival delays are calculated based on: • Specified Transaction Rate • Specified Distribution • There are TWO types of transactors • Coordinated (All users cooperate to deliver total transaction rate) • Independent (All users independently generate their own transaction rate) • Number of transactions controlled by Iterations • Coordinated: number is the total for the whole group • Independent: number is for each user

25. Transactor Properties Choose basic type Specify rate Choose distribution Specify # of Transactions Define Transaction by Scenario Name

26. Transactors Types - Independent • Similar to normal user scheduling • Each user independently executes the transactions, paced by the Transactor • Adding users multiplies the overall transaction rate applied to the server

27. Transactors Types - Coordinated • Used for running transaction rates (not number of users) • All users cooperate to achieve the total transaction rate • A sync point is automatically inserted at the beginning of the Transactor • When Total Rate is selected, adding users will not increase the transaction rate

28. Transactors - Understanding Inter-arrival Delays • Inter-arrival delays are from the BEGINNING of Transaction N-1 to the beginning of Transaction N • Inter-arrival delay is performed prior to the first item in the Scenario that defines the transaction • There is no inter-arrival delay between items in the Scenario • There is an inter-arrival delay even for the first Transaction performed by the Transactor • Calculation is independent of transaction time

29. Transactors - “Falling Behind” • If transaction time EXCEEDS inter-arrival delay • Next transaction starts immediately • This continues until transactor “catches up”

30. Transactors - Transaction Rate Distributions • Constant • Typically unrealistic of user behavior • Avoid unless you really want “beat of the drum” transactions • Negative Exponential • Default • Typical “customers arriving at the ticket counter” distribution • Uniform • Requires % range specification (0% == constant distribution) • Delay is uniformly distributed within the range • [ 1/Rate * (1 - %range/100), 1/Rate * (1 + %range /100)] • Example: Rate = 10/minute, % range is 25% • Then Average Delay = 1/Rate = 6 seconds • Delays will be uniformly distributed between 4.5 and 7.5 seconds

31. Transactor Scheduling Tips - Achieving Rates • Make sure rate is actually achievable! • Server transaction response time << 1/Rate • Add more users to reduce effective per-user transaction rate • Of course you can do nothing when you reach the server’s transaction capacity!

32. Coordinated Transactor Scheduling Tips • To modify the total rate without editing the schedule: • Select User Rate to specify rate on a per-user basis • On Run Schedule dialog, specifying the # of users will control total rate proportionally (rate * number of users assigned) • Make sure the rate is actually achievable for each user • Remember that a Coordinated Transactor contains an automatically embedded Synchronization Point • It must be scheduled deterministically -- don’t put it in a random selector!

33. Transactor Scheduling Tips - Transaction mix • Question: How do I use transactor pacing with a mix of different transactions? • Solution: Use a Random with replacement selector within the Transactor’s scenario • Set Number to select to 1 in the selector (Transaction Iterations need to be set within the Transactor itself)

34. Transactor Scheduling Tips - Mix (Not) • DO NOT use the Random without replacement selector in the scenario of the previous example to achieve a mix of transactions! • With an iteration count of 1, these selectors won’t be random, but will pick the same item each time!!

35. Transactor Monitoring Tips • Actual Rate is the cumulative average (from start) • Actual Rate will take some time to approach Target Rate • First delay is included in the Actual Rate calculation • Sampling rate is asynchronous to the transactor pacing rate • Random distribution will add inherent variance in the Actual Rate • All of these affects are diluted as the number of transactions gets large • Monitoring Independent Transactors is not as straightforward as Coordinated Transactors • Number of users may vary over time • Target Rate and Actual Rate are totaled for all participating users

36. Transactor Monitoring Example Click to get Transactor View Transactor View

37. Transactor Analysis Tips • To see transactor pacing details, view the trace report: User User Group1[1]: Transactor pacing delay of 8982 ms, from 8454 to 17436 User: User Group1[1] Script: Order Beginning timestamp of script Order: 17436 Ending timestamp of script Order: 18037; Duration 601 ms User User Group1[1]: Transactor pacing delay of 14712 ms, from 18037 to 32749 User: User Group1[1] Script: Order Beginning timestamp of script Order: 32788 Ending timestamp of script Order: 33629; Duration 841 ms

38. Mental Break • Questions? • Anyone have a tip to add? • We’re moving on to the next level … • You ready?

39. Miscellaneous Advanced Topics • Tips in this section focus on optimizing, enhancing and extending scripts through VU coding and external C • Level: Intermediate and Advanced

40. VU Performance Tuning - Shared Variables • Shared variable read access • Master: very fast, same as normal variable • Agent: requires round-trip to Master • When using shared variables as global parameters (read-only access) in runs with Agents: • Tip: Copy the shared variable’s value into a local variable, and access the local variable instead (such as in a loop) • Shared Variable write access • Master: implemented with semaphore (medium efficiency) • Agent: requires round-trip to Master • Be judicious in your use of shared variables - don’t go overboard!

41. VU Performance Tuning Tips - Huge Log Files • To reduce size of user log files: • Set Log_level to “Error” except when debugging • To reduce size of user record files (new in Rational Suite PerformanceStudio Version 2000.02.10): • If you don’t need to measure every emulation command, use timers for selected measurements and set Record_level to: • “Timer” - doesn’t write out records for emulation commands (just timers) • “Error” - writes out records for timers and failed emulation commands only

42. VU Performance Tuning Tips - Memory Usage • If you don’t need to match on the _response or log all the data (or only need the first part of the _response ): • Set Max_nrecv_saved to a low number just big enough, such as 1 or 100 • SQL: limits # of rows saved • HTTP, socket: limits # of bytes saved • Applies to all receive commands, not just the “nrecv” flavor • Can be used selectively, such as around large SQL queries, or web pages with lots of gif’s ...

43. Example: Max_nrecv_saved • Using Max_nrecv_saved to only save the first row of a large SQL query: sqlexec ["exec big query"] "select * from books"; pushMax_nrecv_saved = 1; sqlnrecv ["retrieve big query"] ALL_ROWS; popMax_nrecv_saved;

44. VU Performance Tuning Tips - CPU Overhead • Whenever possible use built-in VU functions (such as the string functions) instead of writing custom logic in VU • If you write custom VU code that is processed often: • Write the logic in C and compile it into a shared library (.dll on NT, .so on Solaris) • Call the C routine from VU using VU’s external C linkage

45. VU Performance Tuning - _response processing • Avoid writing custom VU code unnecessarily • Use http_find_values() to extract session ids, etc. • New in Rational Suite PerformanceStudio Version 2000.02.10 • Use match() for any pattern matching • Don’t use “.*” at the beginning of regular expressions (it is unnecessary and adds considerable overhead) • If the match() regular expressions are simple, replace with VU string routines (such as strstr() or subfield())

46. Parameterizing HTTP User IDs & Passwords • Question: How can I have different user ids and passwords used for the various virtual users on an HTTP playback which used Basic Authorization? • Solution: built-in VU base64_encode() function • Replaces external C based solution • “Authorization: Basic” data is base-64 encoded • Replace encoded text values with data from base64_encode() function

47. Example: HTTP Basic Encode #include <VU.h> { string auth_str; auth_str = base64_encode(“mylogin” + “:” + “mypassword”); if (auth_str == “”) user_exit(1, “Can’t convert login/password\n”); rational_com_80 = http_request[“HTTP 003”] “rational.com:80”, HTTP_CONN_DIRECT, “GET / HTTP/1.0\r\n”, … “Authorization: Basic “ + auth_str + “\r\n” “\r\n”; ... }

48. Using External C with Agents • Question: Why can’t the external C shared library be found using an Agent when it works fine on the Master? • Solution: you must place shared library (.dll, .so) on the Agent • External C libraries are not automatically copied to the Agent (and this wouldn’t work anyway for Unix agents) • The proper location is: • <LOCAL_DIRECTORY>\perfstudio_0\externC • <LOCAL_DIRECTORY> is determined based on (in order) • Local Directory set in Computer Settings of Schedule for the Agent in question • The value of the Agent’s environment variable “TMPDIR” • The value of the Agent’s environment variable “TMP” • The value of the Agent’s environment variable “TEMP” • “/tmp” (Unix/Linux only)

49. Locating the Local Directory on Agents • For Agents, external C shared libraries need to be stored under the same root location as where temporary copies of user log files, datapools, and compiled scripts are stored • Location specified with Local Directory field

50. Questions?More tips?