1 / 48

Making Workloads Work in PATROL-Perform, PATROL-Predict and Visualizer

Making Workloads Work in PATROL-Perform, PATROL-Predict and Visualizer. Presentation Overview. Motivation for the Workload Construct Who/How to Design Workloads Case Studies One node, one application, capacity planning study

lilly
Download Presentation

Making Workloads Work in PATROL-Perform, PATROL-Predict and Visualizer

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Making Workloads Work in PATROL-Perform, PATROL-Predict and Visualizer

  2. Presentation Overview • Motivation for the Workload Construct • Who/How to Design Workloads • Case Studies • One node, one application, capacity planning study • Multiple nodes, multiple layers, web application, reporting and capacity planning • Multiple nodes, multiple layers, multiple entry points, web application, capacity planning • Use PATROL-Perform/Predict and Visualizer

  3. Workloads are the Solution • Distributed Systems are too distributed! • Many different processes are part of one “system” • Applications often divided into tiers (“layers”) • Many invocations of an individual processes • Many nodes, a variety of types (UNIX, NT, multi-processor, etc.) • Many nodes in one tier of an application system

  4. Workloads are the Solution • Business objectives are usually simple, e.g. “We need to support 10,000 users” • How can business objectives be imposed on Distributed Systems? • WORKLOADS (as implemented in PATROL-Perform and Predict

  5. Who Should Design Workloads? • Usually, a combination of skills is required • PATROL-Perform/Predict specialist • Performance analyst familiar with the application and environment • Analyst familiar with the application’s software architecture

  6. What Information Do They Need? • A number of data sources are recommended: • Operational installation of logging collectors (system and RDBMS where applicable), Perform, Predict, and Visualizer • Examples of previous performance analysis activities • Additional data regarding loading of the application • A performance reporting or modeling objective

  7. Case Study 1: Vantive Customer Support System • Vantive DB Server environment will change: • Vantive clients will be removed, leaving dedicated DB server • CPU will change from HP K570 (6-way) to an HP N4000 (4-way) • Memory will change from 8 GB to 4 GB • What growth can be supported on the proposed configuration? • Note: This case study is part of BMC Training Classes

  8. Case Study 1: First Look at the Data using Analyze • Can use Analyze “browse” or use Manager/Visualizer if data might be complex

  9. Case Study 1: First Look at the Data using Analyze • 80/20 Rule: Use Process Detail Accumulated by Process report to see what’s important

  10. Case Study 1: First Look at the Data using Analyze • The important categories are oracle, iwserver, Patrol, and bgs • Multiple oracle instances can be subdivided

  11. Case Study 1: First Look at the Data using Analyze • Isolate the oracle instances until we find out what’s important • Isolate performance tools by vendor

  12. Case Study 1: Workload Strategy • For iwserver and oracle, use “COUNT-BY PROCESS” attribute • Makes transaction count an actual load indicator for client and server processes • Multiple independent transactions (e.g. oracle and perf_products) provide two levels of reporting in Visualizer (and Predict): transaction and workload • Multiple oracle categories specified using precedence

  13. Case Study 1: Workload Details CREATE TRAN-CLASS bmc_products INDEPENDENT YES PROCESSES Patrol.* CREATE TRAN-CLASS bgs_products INDEPENDENT YES PROCESSES bgs.* PROCESSES best1.* CREATE TRAN-CLASS hp_products INDEPENDENT YES PROCESSES .* "/opt/perf/bin.*" CREATE TRAN-CLASS iwserver INDEPENDENT YES PROCESSES iwserver COUNT-BY PROCESS 1.000000 CREATE TRAN-CLASS oracle_vancs8p INDEPENDENT YES PROCESSES oracle ".*VANCS8P.*" COUNT-BY PROCESS 1.000000 CREATE TRAN-CLASS oracle_vanptrkp INDEPENDENT YES PROCESSES oracle ".*VANPTRKP.*" COUNT-BY PROCESS 1.000000 CREATE TRAN-CLASS oracle_vantl8cp INDEPENDENT YES PROCESSES oracle ".*VANTL8CP.*" COUNT-BY PROCESS 1.000000 CREATE TRAN-CLASS oracle_other INDEPENDENT YES PROCESSES oracle COUNT-BY PROCESS 1.000000 CREATE WKLSPEC perf_products TCLASS-LIST bmc_products TCLASS-LIST bgs_products TCLASS-LIST hp_products CREATE WKLSPEC vantive_appl TCLASS-LIST iwserver CREATE WKLSPEC vantive_db TCLASS-LIST oracle_vancs8p CREATE WKLSPEC oracle_other TCLASS-LIST oracle_vanptrkp TCLASS-LIST oracle_vantl8cp TCLASS-LIST oracle_other

  14. Case Study 1: Is the 80/20 Rule Satisfied? • Check categories of interest to see if they have what’s expected • Check that “zzz” is sufficiently small

  15. Case Study 1: Is the 80/20 Rule Satisfied? • Observe transaction counts for categories of interest to see if they have what’s expected

  16. Case Study 1: Big Picture View of the Data • If a lot a variability exists in the data, try out one or two more intervals and check the results • Use Manager to bulk process data, using Analyze and Predict to generate Visualizer info • Select appropriate Visualizer intervals • One hour for general capacity planning • As small as one spill for performance analysis • Choose Visualizer graphics according to the objectives of the performance study (reporting, modeling, etc.) • Graphics for each resource (CPU, Memory, I/O) • Graphics to relate transactions and workloads to system resources

  17. Case Study 1: Workload CPU Utilization in Visualizer • Vantive_db is the largest consumer • Removing vantive_appl will provide little relief • Perf_products cost/benefit should be investigated • High workload variability indicates need for multiple baseline models

  18. Case Study 1: Transaction CPU Utilization in Visualizer • Additional detail compared with Workload CPU Utilization • Within oracle_other, oracle with no instance is dominant • Within perf_products, bmc (patrol) is dominant

  19. Case Study 1: Workload Transaction Rate in Visualizer • Relative load indicated by number of UNIX processes • Vantive_appl is small, so it’s removal will not save much memory • Oracle_other processes are numerous, so it’s memory impact should be explicitly modeled • Vantive_db and oracle_other track together, so can be combined in the modeling phase

  20. Case Study 1: Workload Response Time in Visualizer • Vantive_db is the important contributor, so the capacity planning study will focus exclusively on that workload

  21. Interval Vantive _db CPU Vantive _db I/O Vantive _db #users Vantive _appl CPU Vantive _appl I/O Vantive _appl #users Vantive total # procs System CPU queue 20 Sep 12-13 7.70 2.25 555 2.20 .04 278 3800 5.9 21 Sep 10-11 9.01 2.85 628 2.61 .10 285 4228 7.0 22 Sep 10-11 8.52 2.09 589 2.27 .05 265 3280 3.3 Case Study 1: Workload Modeling Strategy • Workloads/transactions can be simplified for the modeling phase • Oracle can be combined into one workload, vantive_db • Only one workload to manipulate, one workload response time to track • Use Visualizer info to select appropriate baseline modeling intervals • Workload volume, CPU, and I/O variations studied • Three baseline models constructed

  22. Case Study 1: Workload Modeling Results • Vantive_appl is removed • Vantive_db is grown in Scenario Planner • Results are viewed in Visualizer • Vantive_db response time checked for meeting objective • CPU utilization checked for meeting objective • Resulting workload volume recorded for each baseline

  23. 20 Sep 20 Sep at 11 21 Sep at 10 21 Sep 22 Sep at 10 22 Sep 23 Sep at 16 24 Sep at 10 Average Average 943 2220 1072 2410 904 2490 916 961 2373 960 Case Study 1: Workload Modeling Results • Results from the 3 what-if models compared with measured baseline volumes give growth that can be sustained • 2373 / 960 = 2.47 or about 150% growth What-if Baselines

  24. Case Study 2: Multi-Node, Multi-Tier Web Application • Web application is not in production, but has been benchmarked twice • Benchmarks can be compared to see changes in implementation • Model(s) from benchmark measurements can be used to predict configuration requirements for a given user load • Benchmark environment means that external loading parameters are available e.g. web hits and number of users

  25. Case Study 2: Web Application Structure USERS Web Server Layer 01 02 03 04 05 06 07 08 09 10 Content Server Layer 01 01 01 02 03 04 05 06 Database Server Layer 01 02

  26. Case Study 2: First Look at the Data using Analyze • Analyze “browse”

  27. Case Study 2: First Look at Data with Manager/Visualizer • Select only relevant intervals for Manager to process for Visualizer (7 15-minute spills) • Review layers for consistency within the layer • Use Commands Hierarchy for 80/20 Rule analysis

  28. Case Study 2: First Look at the Data using Manager/Visualizer

  29. Case Study 2: First Look at the Data using Manager/Visualizer java java-wrapper ns-httpd oracle

  30. Case Study 2: Workload Strategy • A single workload with a single independent transaction calling multiple dependent transactions • Workload response time is the sum of all layers • Makes modeling simple: only one workload • Maintains multiple reporting levels: transactions and workload • For each layer, use “COUNT-BY TRANSACTION” attribute • Use externally supplied load indicator “web hits” • Makes transaction count an actual load indicator

  31. Case Study 2: Workload Details independent dependents

  32. Case Study 2: Workload Details independent dependent Specify nodes for dependent Uncheck Distribution options

  33. Case Study 2: Workload Strategy • Strategy for choosing externally measured load indicator: compare observed resource utilization with candidate load indicators

  34. Case Study 2: Workload Strategy • Determine value for “web hits” for each spill • Use one .an file for each combination of interval and transaction count • “COUNT-BY TRANSACTION” specification is per node (so divide the total hits in the interval by the number of nodes in each layer) • Independent transaction layer must have “COUNT-BY TRANSACTION” in order to yield response time per web hit in the model, dependent transaction layers are optional • Generate Analyze and Predict .vis files and make a separate database

  35. Case Study 2: Workload Details CREATE TRAN-CLASS oracle INDEPENDENT NO DISTRIBUTE-BY-NODE NO DISTRIBUTE-BY-USER NO FRACTION_TO_DISTRIBUTE 1.000000 PROCESSES oracle COUNT-BY TRAN-CALLS 106963.000000 ALLOCATE-BY TRAN-COUNT CREATE TRAN-CLASS ns-httpd INDEPENDENT NO DISTRIBUTE-BY-NODE NO DISTRIBUTE-BY-USER NO PROCESSES ns-httpd COUNT-BY TRAN-CALLS 21393.000000 ALLOCATE-BY TRAN-COUNT CREATE TRAN-CLASS java INDEPENDENT YES PROCESSES .*java.* COUNT-BY TRAN-CALLS 35654.000000 INITIATES oracle NODE-LIST nyprddsmdbc02 NODE-LIST nyprddsmdbc01 INITIATES ns-httpd NODE-LIST nyprdwsmwss01 NODE-LIST nyprdwsmwss02 … web hits for 15 minutes 106963 * 2 = 213926 21393 * 10 = 213930 35654 * 6 = 213924

  36. Case Study 2: Workload Response Time in Visualizer • web_app includes all layers, so the whole picture is here Response Time per Web Hit Web Hits/Hr

  37. Case Study 2: Workload Modeling Strategy • All modeling is accomplished via the single workload web_app, parameterized as web hits/hour • Scenario planner used to specify growth for web_app • Only one workload to manipulate, one workload response time to track • Use Excel to determine relationship between users and web hits User = 325 web hits/hour

  38. Case Study 3: Multi-Node, Multi-Tier, Multi-Entry Web Application • Four-tiered web application • Users may enter at any of three layers • All tiers below the entry point are accessed

  39. Case Study 3: Web Application Structure Mobile Phone Server Layer USERS 01 02 02 02 02 03 03 08 09 10 01 05 04 Web Server Layer USERS 01 02 03 Content Server Layer USERS 01 01 02 02 03 03 04 04 05 06 Database Server Layer 01

  40. Case Study 3: Workload Strategy • Use external counts of users at each layer as “COUNT-BY TRANSACTION” specification • All layers must have “COUNT-BY TRANSACTION”, but there are different totals at each layer • One independent transaction/workload is needed for each entry point • Dependent transactions called by multiple independent transactions

  41. Case Study 3: Workload Details independent dependents

  42. Case Study 3: Workload Details workload independent (with no data)

  43. Case Study 3: Workload Details CREATE TRAN-CLASS SQL_Layer DESCRIPTION Dependent SQL Transaction OP-SYSTEM NT INDEPENDENT NO DISTRIBUTE-BY-NODE NO DISTRIBUTE-BY-USER NO COUNT-BY TRAN-CALLS 30000.000000 ALLOCATE-BY TRAN-COUNT CREATE TRAN-CLASS Application_Layer DESCRIPTION Dependent Applications Transaction OP-SYSTEM NT INDEPENDENT NO DISTRIBUTE-BY-NODE NO DISTRIBUTE-BY-USER NO COUNT-BY TRAN-CALLS 5000.000000 ALLOCATE-BY TRAN-COUNT CREATE TRAN-CLASS Web_Layer DESCRIPTION Dependent Web Transaction OP-SYSTEM NT INDEPENDENT NO DISTRIBUTE-BY-NODE NO DISTRIBUTE-BY-USER NO COUNT-BY TRAN-CALLS 5000.000000 ALLOCATE-BY TRAN-COUNT 30000 users 30000 * 1 = 30000 DB 5000 * 6 = 30000 APP 5000 * 3 = 15000 WEB

  44. Case Study 3: Workload Details CREATE TRAN-CLASS Entry_at_Nokia_Layer DESCRIPTION Independent Nokia Transaction OP-SYSTEM NT INDEPENDENT YES PROCESSES WapSmoke COUNT-BY TRAN-CALLS 2500.000000 INITIATES Application_Layer INITIATES SQL_Layer INITIATES Web_Layer CREATE TRAN-CLASS Entry_at_Web_Layer DESCRIPTION Independent Web Transaction OP-SYSTEM NT INDEPENDENT YES PROCESSES Wuser32 PROCESSES BRAD32 COUNT-BY TRAN-CALLS 1667.000000 INITIATES Application_Layer INITIATES SQL_Layer CREATE TRAN-CLASS Entry_at_App_Layer DESCRIPTION Independent Applications Transaction OP-SYSTEM NT INDEPENDENT YES PROCESSES clisvcl COUNT-BY TRAN-CALLS 2500.000000 INITIATES SQL_Layer 2500 * 4 = 10000 NOKIA 1667 * 3 = 5000 WEB + 2500 * 6 = 15000 APP ------------------------------------ 30000 USERS

  45. Case Study 3: Workload Results Users by Entry and Total Response Time by Entry Point

  46. Workload Tools Summary • Knowledge of the application • Analyze Process Detail Accumulated by Process report • Analyze Process Tree (in the .anr file) • Visualizer Commands Hierarchy (CPU Utilization and/or process count) • Auto-discovery workload tools • Statistical correlation • Process name discovery

  47. Workload Design Summary • Knowledge of business objective • Allows appropriate reduction in complexity • Determines what metrics are important • Determines what data is necessary

  48. Workload Design Summary • Analyze/Predict Workload construct is flexible • Workload consists of one or more independent transaction(s) • Independent transaction may call as many dependent transactions as desired • Use node lists to further specify where transactions should be found • Can use user classes in conjunction with transaction classes • Workload precedence can be used as a filter • When defining transaction/user classes, use pattern matching in process/user name specifications for clarity and simplicity

More Related