NCCMG November 4th, 2003 Catherine Liu Applied Expert Systems, Inc. Menlo Park, California

1. It’s All About Performance! NCCMG November 4th, 2003 Catherine Liu Applied Expert Systems, Inc. Menlo Park, California Permission is granted to NCCMG to distribute images of the slides of the presentation to attendees and members. All other rights reserved.

2. Agenda • The eBusiness Building Blocks • The eBusiness Performance Focus • The eBusiness Performance Management Process • ePerformance Monitoring • ePerformance Reporting • Conclusions

3. eBusiness is “The Business” • Online sales surged 48% in 2002 • 70% of retailers reported positive operating margins from Web sales • Compares to 56% profitability in ‘01 • “Brick & Mortar” Co’s welcome these positive eBusiness operating margins • Web sales often able to offset business losses from “offline” operations

4. eBusiness is “The Business” • Most new user applications will be Web-based designs • Intranet/Internet is now accepted “norm”, or standard access method • $1 trillion+ in existing mission-critical apps still need enhancements or rewrites to become Intra- or Internet accessible

5. Servers Routers Switches Hubs Network Connections Channel Links MVS Appl. Servers VTAM Buffers VTAM Buffers CSM Clients User Apps TCP/IP VTAM TCP/IP World Wide Web The eBusiness Building Blocks

6. TCP/IP Routes Web The Ring of eBusiness

7. The eBusiness Challenge • Managers and analysts still rely on the heritage of SNA-based measurements • Different performance “pulse” for sessions & services of eBusiness apps • Different “skeletal” framework for eBusiness customers and connectivity • Yet, Intranet/Internet networking model needs similar performance measurement & management

8. It’s All About Performance • It’s easy to build a Web-site • It’s not so easy to build a site with… • Balanced Content • Good Human Interaction • Acceptable Performance

9. It’s All About Performance • Many businesses rely on fast, instant response to stay competitive, such as… • E-Trade • eBay • Prefer to simplify Web-site design • Limit page clutter • Don’t load too many gifs • Avoid extra actions and large files

10. It’s All About Performance • But common-sense ePerformance techniques only go so far… • Performance industry must focus on… • The implementation of an ongoing ePerformance Solution • The improvement and assurance of eBusiness Service Levels

11. Focus on ePerformance Monitor and Report on: • Web and other eBusiness Applications • End-to-End Service Level monitoring • Response Times for Web transactions • TCP/IP Performance • End-to-End network response times • Effects of operational performance control • Network Routes and Equipment • End-to-End route response times • Identify route patterns and defects

12. End-to-End eBusiness Response Time

13. What is End-to-End (E-2-E) eBusiness Response Time? • The response time for mission critical Web pages or Web transactions • You can measure both Internet or Intranet operations quickly • Can be performed from single or multiple users point of view • Measuring and monitoring this is absolutely doable

14. An ePerformance Web-site SLO Solution Example • Establish easy-to-track, corporate ”Web-Unit” Service Level Objectives • Primary Web Metrics: • Web-site Availability (site up-time) • “Web-Unit” Performance (page load-time) • Need capability to view and analyze each in real-time, as well as historically • Reports must be reflective of the specified Service Level Objectives

15. Basis for Web-site SLAs • Use SLAs strategically for Web sites • External Sites: • Key indicator of eBusiness health and status as the new benchmark for corporate ‘vitality’ • Internal Sites: • Now used for critical corporate functions • Obvious and essential tie-in to revenue, processes and productivity • Tool for exceptions & tuning diagnostics • Foundation for site capacity planning

18. Monitor “load times” for • critical URL’s in real time. • Focus on SLO thresholds for • “web-unit” Response Times.

19. Report on avg. & • minimum response • times for key URL’s • historically. • Focus on daily website • availability for SLA’s.

20. End-to-End Network Response Time

21. What is End-to-End (E-2-E) Network Response Time? • The IP-based network response time from a source device to the destination or, the reverse • IP socket transit time, not necessarily at the application level • Measuring and monitoring this is just as critical to assure Service Levels for user Web transactions can be met

22. End-to-End Route Response Time

23. What is End-to-End (E-2-E) Route Response Time? • Added details of network response time • Time spent by packets as they travel through individual route segments • Necessary to observe over time • Focus on identifying… • Longest routes in terms of hops • Longest segments in terms of response time

25. Steps to ePerformance Management • Initiate Service Level Monitoring • Establish Service Level Objectives • Establish Service Level Agreements • Measure, Analyze and Report • Manage and Maintain

26. Initiate Service Level Monitoring • Key IP stack and network indicators • Channel links to the IP network • CSM and VTAM buffer usage • Services, applications, and ports • Client session activity and failures • End-to-end network availability, performance, and throughput

27. A Sample ePerformance Monitoring Solution Utilize Real-Time Monitoring Alerts • Check IP focal point for health indicators • Check CPU performance of stack & appls • Check CSM buffer usage • Check status and performance of IP links • Check IP application port status • Check session logs for recent activity

29. A Sample ePerformance Monitoring Solution Utilize Near-Time Analysis & Diagnostics • Check for availability/performance alerts, as well as network connectivity events • Review user and session activity • Check hardware status & performance • Verify route status and performance • Isolated monitoring of specific end users • Perform further route or cTrace diagnostics

30. Establish SLOs & SLAs Should include, but not necessarily be limited to, the following: • Web Application Performance • Network Availability • Network Response Time • Route Performance

31. Introducing the I-Unit

32. A Sample ePerformance E-2-E SLO Solution Establish “I-Unit” as an E-2-E Metric • Packet equates to Unit of Data that is passed between IP & network interface • Inter-Network Unit of Data = I-Unit • Focus is on consistent measurement of corp. network response time for I-Unit • Business entities can set specific I-Unit based on analysis & customer consensus

33. Basis for ePerformance E-2-E SLAs Then standardize with I-Unit SLAs • Easy to implement • Consistently measurable • Good for inside users or outside customers • Use as accountability tool for External Service Providers (ISPs, ASPs) • Tool for exception and tuning diagnostics • Establish baseline for capacity planning • Foundation for costing and charge-back

34. Enterprise Network Service Level Measurements

36. Batch/PR (Batch/Performance Reporter) Date: 12/16/2002 REPORT: BatchPRSLS A1 Special Company DATE: 2002.350 SHIFT: 1 FROM Hour: 9 TO Hour: 15 NETWORK AVAILABILITY RATING Total # of Critical Resources Monitored: 16 Total # of Observed Events: 4012 Total # of Resources Unavailable Events 292 Percentage of Network Availability: 92.722% Top 3 Critical Resources Reporting Unavailable Events Resource IP Address # Events # Unavailable Percentage --------------------- ---------------- ----------------- -------------- 137.72.43.244 991 85 8.577% 137.72.43.21 260 81 31.154% 137.72.43.141 371 65 17.520% NETWORK RESPONSE TIME RATING Total # of Critical Resources Monitored: 16 Total # of Observed Events: 4012 Total # of Threshold Exceeding Events: 123 Percentage of Network RT within Threshold: 96.934% Top 3 Critical Resources Exceeding RT Threshold: Resource IP Address # Events # Over Threshold Percentage --------------------- ---------------- ----------------- -------------- 216.32.74.53 282 44 15.603% 137.72.43.241 319 36 11.285% 137.72.43.242 284 33 11.620%

37. Reporting

38. The ePerformance Management Process Measure, Analyze and Report • Report Routinely on Key Metrics • Timeframe-focused (shift, daily, weekly…) • Management-level summaries • Technical-level details • Workload (sessions and traffic) • Performance (I-Unit, FTP throughput…) • Exceptions (peaks, valleys, hangs, failures…)

39. ePerformance IP Workload Reporting Solution Example Implement Weekly IP Activity Reports • Report by weekly timeframes • Report by session totals and data traffic • Selectively by specific application • Selectively by IP addresses or DNS clients • Identify top users and busiest applications • Hardcopy batch scheduled reports

41. Batch/PR Workload Server Application Summary Date: 12/21/2002Report: WLServer A1 Special Company Date From: 12/16/2002. To: 12/20/2002. Days of Week included: Monday Wednesday Friday Server Application Groups Name # Ports Sessions Total Bout BOut Max BOut Avg Total Bin BIn Max Bin Avg TELNET 23 17 9693810 3485159 570224 3155902 2229102 185641 TELNET 1023 0 0 0 0 0 0 0 Total 17 9693810 3485159 570224 3155902 2229102 185641 FTP 20 4 19652 5840 4913 0 0 0 FTP 21 10 4868 715 486 901 131 90 FTP 1020 0 0 0 0 0 0 0 FTP 1021 0 0 0 0 0 0 0 Total 14 24520 5840 1751 901 131 64 DB2 6001 472 113945 21176 241 29694 154 62 Total 472 113945 21176 241 29694 154 62 DBMOVER 6930 0 0 0 0 0 0 0 Total 0 0 0 0 0 0 0

42. Specific application reporting

43. A Sample ePerformance FTP Activity Reporting Solution Utilize Key FTP Reports & Auditing • Selectively by time frames • Selectively by user ID, dataset, IP address • Selectively by Server or Client sessions • Identify session totals and traffic • Identify FTP abuses, hangs and failures • Utilize FTP session logs • Run cTrace for problem FTP’s & packets

45. Highly Detailed Diagnostic Information

47. cTrace database analysis zoom-in to specific trace record

48. The ePerformance Management Process Manage and Maintain: • Real-time monitoring, Sysplex-wide • Near-time analysis & diagnostic tools • Valid alerts & thresholds for key criteria • Consistently measurable SLOs • Historical reporting methodology • Historical trending capability

49. The ePerformance Management Solution Manage Key ePerformance Subjects • Multiple stacks simultaneously • Multiple TCP/IP levels simultaneously • Multiple thresholds and alerting methods • Interval-based and/or shift specific • Enterprise-wide for IP resources • Management summaries or detail specific

50. The ePerformance Maintenance Solution Maintain Key ePerformance Metrics • Event-based, SMF data (Type 118/119) • Interval-based, SMF data (Performance and Workload) • TCP/IP session log activity • Alert history and command audit logs • Tracerte data collected automatically • CSM and VTAM buffer usage history • SNMP MIB data (Enterprise-wide)