Availability Monitoring Using ASAP MA-10-HP

1. Availability Monitoring Using ASAPMA-10-HP Bob Rojewski and Collin O’Brien Software Designers, HP bob.rojewski@hp.com collin.obrien@hp.com http://www.NonStopASAP.com

2. Agenda • ASAP Overview • ASAP Status Update • Futures: ASAP 2.5 NonStopAsap.com

3. ASAP Overview NonStopAsap.com

4. What Is ASAP? • Availability Monitor • State Monitor • Performance Monitor • Fault-Tolerant Infrastructure To Support And Enhance Monitoring Functions • Not An Enterprise Management Framework, ASAP interfaces with various open EMF’s NonStopAsap.com

5. ASAP Performance & State Monitoring • Monitors performance & state of system components, including CPUs, disks, processes, etc. • Includes all state and performance-related attributes of each object (usage percentages, rates, status, …) • Allows thresholds to be specified for attributes; if an object/attribute exceeds a threshold, an alert will be generated • Can be extended to applications via ASAPX API NonStopAsap.com

6. ASAP Availability Monitoring • Performance and state monitoring make a quantitative statement about objects • Availability monitoring makes a user-defined, qualitative statement about objects • Can define a rich set of service level objectives for monitored objects and attributes • Alerting infrastructure driven off of objectives • Applications can utilize this capability via ASAPX API; application service level objectives are defined and handled in the same way as system objectives NonStopAsap.com

7. SharedMemory ASAP Consumer Provider ASAP - Extension API ASAPX ASAP_REGISTER_( domain_name : length , segment_offset ); . . ENDTRANSACTION; ASAP_UPDATE_( segment_offset, error_detail , Data_item, Value, Math ); ASAPX Data_item- Defined by EDL Value - Defined by application Math = 0 - Add Value to Data_itemMath = 1 - Assign Value to Data_itemMath = 2 - Assign Text to Data_item ASAPDatabase NonStopAsap.com

8. ASAP Infrastructure • Distributed monitoring environment • Centralized historical database • Fault-tolerant, massively parallel architecture for scalability and low overhead • Wide variety of published and open interfaces: NonStopAsap.com

9. ASAP Status Update NonStopAsap.com

10. ASAP 2.4 – Status Update • ASAP core software monitors the following subsystems: • ASAP Extension (ASAPX) allows applications to utilize ASAP monitoring infrastructure • Third-party plug-ins provide monitoring for additional subsystems: NonStopAsap.com

11. ASAP 2.4 – New Features • ASAP 2.4 focuses on improved scalability: • Process ZLE grouping and aggregation of abstract process domains • Support for 10,000 abstract domains per processor • New in-memory detail and aggregation functions NonStopAsap.com

12. ASAP 2.4 – Scalability Can you monitor? 10,000 processes in a processor? 160,000 processes in a system? processes in an Expand network? 40,800,000 • That’s what’s coming! Super-fast, super-scalar systems • Requirements: • Must analyze each individual process and compare against predefined service levels, alerting when necessary • Must be able to group processes and view at the group level • Must be able to aggregate/propagate data and alerts to the group level • Must be able to set service level objectives at aggregate levels • Must be able to store summary information about the group for historical purposes • Must have access to detail information when needed NonStopAsap.com

13. ASAP: Examines each individual monitored process at each interval Computes and analyzes over two dozen attributes for each process Compares each state-pair attribute against pre-defined objectives Generates EMS and/or ASAP alerts when objectives aren’t met Stores historical information about each process in the ASAP database Provides alert, status and/or performance data to a variety of optional user interfaces. ASAP 2.4 – Process Monitoring MONITOR PROCESS $A $C $B $A $F $D $E $A $B $C $D $E $F NonStopAsap.com

14. ASAP 2.4 – Process ZLE • Hierarchical Process Grouping • Allows logical groups of processes • For example group all servers for the sales application • Or group all spooler processes • Aggregation/Propagation to Group Levels • Provides aggregate summaries and alerts • Real-time historical data reduction • Configuration Options • 5 Levels of hierarchical naming • Aggregate domains at any level • Control of aggregate/propagate function MONITOR PROCESS SALES\$A Sales $C $B $A $F $D $E Sales NonStopAsap.com

15. The MONITOR Command Defines Process Domains MONITOR PROCESS $ABC MONITOR PROCESS SALES\ORDERS\SERVERS\$ABC MONITOR PROCESS SPOOLER\SUPERVISORS\$SPLS Benefits of Process Groups Hierarchical views and state propagation Aggregate summaries at each level More manageable navigation and process monitoring Monitor the service and the individual components Process Domain Name Rules Up to 64 bytes in length Up to 5 hierarchical levels, separated by a backslash (“\”) Last level denotes process name in abstract group ASAP 2.4 – Hierarchical Process Grouping NonStopAsap.com

16. Use the MONITOR command to create aggregate domains MONITOR PROCESS SALES\ORDERS\SERVERS\# MONITOR PROCESS SALES\ORDERS\# MONITOR PROCESS SALES\# Each MONITOR PROCESS <name>\# command instructs ASAP to create an aggregate record of that name The aggregate record contains attribute values that are computed across all group members at and below that level in the name hierarchy Attribute values in the summary records are determined by Data Aggregation or Attribute Propagation Attribute states in the summary records are determined using State Propagation or an objective comparison ASAP 2.4 – Aggregate Domains NonStopAsap.com

17. Data Aggregation ASAP combines attribute values using a mathematical comparison or computation across all members of the group ASAP uses SUM, AVG, MIN, MAX or CNT functions to determine the attribute value in the aggregate record Attribute Propagation If no Data Aggregation is specified for an attribute ASAP propagates an attribute value to the aggregate record The attribute value with the worst (highest) ASAP state gets propagated to the aggregate record State Propagation ASAP propagates the worst (highest) ASAP state to the aggregate record unless an objective is set on the aggregate domain attribute ASAP 2.4 – Aggregate Domains NonStopAsap.com

18. ASAP 2.4 – Aggregate-Only Domains • The MONITOR Command Defines Aggregate-Only Domains • MONITOR PROCESS SALES\ORDERS\## • MONITOR PROCESS SALES\## • Aggregate Only • Process each detail record • Combine data into an aggregate record • Write ONLY the aggregate record to the ASAP database • Store the data from hundreds or thousands of processes in a single historical record • Detail Records Can Still be Seen ! • PROCESS, MEMORY option retrieves detailed process information directly from ASAP memory NonStopAsap.com

19. ASAPX 2.4 – Scalability Can you monitor? 10,000 application domains in a processor? 160,000 domains in a system? 40,800,000 domains in an Expand network? • That’s what’s here today in ASAPX 2.4! • Requirements: • Must analyze each individual domain and compare against predefined service levels, alerting when necessary • Must be able to group domains and view at the group level • Must be able to aggregate/propagate data and alerts to the group level • Must be able to set service level objectives on aggregate data • Must be able to store summary information about the group for historical purposes • Must have access to detail information when needed NonStopAsap.com

20. ASAPX 2.4 – ASAPX ZLE • Hierarchical Process Grouping • Aggregation/Propagation to Group Levels • 10,000 domains per CPU • 10,000 aggregates per CPU • Control of aggregate function • DataItem and S operands now have optional aggregate codes • #<n> [SUM | AVG | MIN | MAX | CNT] • S [SUM | AVG] • APP, MEMORY option retrieves detailed process information directly from ASAP memory ASAP_REGISTER_ Sales Sales\ Sales\ Sales\Inv Sales\ Sales\ Sales\ Sales NonStopAsap.com

21. ASAPX 2.4 – Other Enhancements • New ASAPX Dynamic Link Library • ASAPXDLL • Compiled from same code base as all other libraries • Use with PIC applications • New DataItem Type • DATAITEM “<n> T” • Specifies DataItem is 8-byte ASCII text string • Should use MATH 2 (replace text) • Should not use AVG or SUM aggregate controls NonStopAsap.com

22. Futures: ASAP 2.5 NonStopAsap.com

23. ASAP 2.5 – Alert-based Actions • Allows ASAP to automatically take actions when objectives are not met • Actions are defined with the associated RANK objective • Can be specified for any objective and object (both system and user applications) • Can control how often each action is invoked NonStopAsap.com

24. ASAP 2.5 – Alert-based Actions • Actions can invoke any NSK command interpreter (TACL, FUP, …), macro, or user program • Token indicators in action definition allow entity name, state, domain, attribute, value, last state, operation, and goal of alerting object to be passed to CI, macro, or program • Examples: RANK FILE $SYSTEM.SYSTEM.USERID, RWEP = "OOOO“ ACTION "FUP SECURE <#DOMAIN>, '<#GOAL>‘ " RANK CPU, STATUS ACTION "TACL RELOAD <#DOMAIN>, PRIME" WAIT 15 REPEAT RANK MYAPP TRADING, TRANSRATE > 100 ACTION “MYAPPMGR <#DOMAIN>, <#ATTR>, <#VALUE>, <#GOAL>” NonStopAsap.com

25. ASAP 2.5 – Historical Data Management • New DBMaxDays option controls how much data is retained • Different retention periods can be specified for each entity type (e.g. save 30 days of CPU data, 6 months of APP data, …) • Works in conjunction with SET RETAIN ROLLOVER • ASAP automatically manages historical data file sets, and purges file sets that are no longer needed COMM CPU TMF CPU APP APP TMF TMF APP APP SPOOL FILE NonStopAsap.com

26. ASAP 2.5 – Email Notifications • ASAP alerts can optionally trigger email notifications • Can send specific alert info (cell phones, pagers) ASAP Critical Alert: \Newyork\Disk\$Data15 Mirror Down • Can send full state summary and state change info (email, management reporting) NonStopAsap.com

27. ASAP 2.5 – Email Notifications • Emails may be issued when state changes occur, at preset times/intervals, and/or at every sample • Can be sent to one or multiple recipients • Content dictated by ASAP client Object State Change (OSC) configuration • Because email notifications are controlled by the client, each operator can customize when notifications are sent and what they contain NonStopAsap.com

28. ASAP 2.5 – Enhanced Communications Monitoring • Extends monitoring to: • Gathers all state and performance data for each line • Automatically discovers and monitors all lines (unless configured otherwise) • Objectives may be set on all key attributes • Fully integrated with ASAP infrastructure NonStopAsap.com

29. ASAP 2.5 – File ZLE • Hierarchical File Grouping • Allows logical groups of files • E.g. group all files for the trading application, or group all spooler data files • Aggregation/Propagation to Group Levels • Provides aggregate summaries and alerts • Combines detail data into an aggregate record • Configuration Options • 5 Levels of hierarchical naming • Aggregate domains at any level • Control of aggregate/propagate function • Monitor OSS files by their OSS path names • Aggregate Only Records • Real-time historical data reduction • Write ONLY the aggregate record to the ASAP database • Store the data from hundreds or thousands of files in a single historical record • Detail records can still be seen; FILE, MEMORY option retrieves detailed files information directly from ASAP memory NonStopAsap.com

30. ASAP 2.5 – Process ZLE, The Sequel • Can now monitor processes by object file name • Support for wildcards in process names • OSS processes can be monitored by specifying the OSS path/object file name • ASAP displays the OSS path/object file name for OSS processes, instead of the Guardian file name • New attributes to track number of processes in a set, and the number alerting $SYSTEM.SYSTEM.SQLCI $ABC /bin/sort NonStopAsap.com

31. ASAP 2.5 – EDL Synchronization • ASAP entity definition (EDL) files reside on both the server and client • Changes to server EDL must be synchronized with client EDL • Prior versions of ASAP detected version mismatches between client and server, but permitted manual synchronization only • New version will allow automated download of server EDL to client • Simplifies use and maintenance, greatly reduces likelihood of errors NonStopAsap.com

32. ASAP 2.5 – Enhanced Event Reporting • Additional tokens now provided in every alert event message: • ZASP-TKN-ASAP-OBJECTNAME contains the object name • ZASP-TKN-ASAP-STATE contains the current state of the object • ZASP-TKN-ASAP-LASTSTATE contains the previous state of the object • Can associate specific event numbers with an objective via the RANK command; one for object transition to exception state (objective not met), one for object transition to normal state • RANK MYAPP TRADING, TRANSRATE > 100 CRITICAL 7001-7002 NonStopAsap.com

33. ASAP 2.5 – OpenView Integration • Enables ASAP data to be added to OpenView • Supports all object types (system, application, third-party) • Shares object state, availability, performance, and service level data • Utilizes both OpenView Services and OpenView Nodes Views • Integration is optional; OpenView is not required in order to deploy ASAP • Included in base ASAP product NonStopAsap.com

34. ASAP 2.5 – OpenView Integration • ASAP data can be accessed via OpenView GUI Console or OpenView Web Interface NonStopAsap.com

35. ASAP 2.5 – Other Enhancements • Client GUI configuration optionally password-protected • SGP CPU placement will allow SGPs to run in any CPU • New “Node Down” Alert can be issued when ASAP detects that a node is down or unreachable • Disk monitoring allows threshold to be set on largest fragment NonStopAsap.com

36. ASAP 2.5 – Release Information • Official release in fall of 2004 • EAP this summer • New Basic Operations Bundle (SA29v1) for S-series will contain this version update upon release NonStopAsap.com

37. Futures: ASAP and Web ViewPoint • Future integration of ASAP and Web ViewPoint will provide: • Web interface to ASAP data • NonStop server accessible from any browser • Object-based view of ASAP alerts within Web ViewPoint object manager NonStopAsap.com

38. For More Information • Background information on ASAP, white papers, downloads of trial versions, and more available at: http://www.NonStopASAP.com • Questions? Email us at: support@NonStopASAP.com NonStopAsap.com

39. NonStopAsap.com