Application Performance Measurement MIB draft-waldbusser-rmonmib-apm-00.txt

1. Application Performance Measurement MIB draft-waldbusser-rmonmib-apm-00.txt Steve Waldbusser Lucent Technologies

2. Solution • 3 Fundamental Types of Applications • Transaction Based • Throughput Based • Streaming

3. Transaction-Based Applications • Application Availability • The percentage of transactions that are successful • Application Responsiveness • The time that passes from when a user submits a request until when they see the results (or a confirmation)

4. Throughput-Based Applications • Application Availability • The percentage of transactions that are successful • Application Responsiveness • The throughput (bytes/sec) seen from the beginning of the transaction to the end

5. Streaming-based Applications • Application Availability • The percentage of transactions that are successful • Application Responsiveness • The percentage of time that the application delivered signal to the user • AKA: The percentage of time that the application kept the playback buffer from emptying

6. APM Metrics Model

7. Transaction Types • Some applications may look transaction-oriented sometimes and transport-oriented other times • Often classifying them by verbs will help • Sometimes you want to have both measurements available • MIB allows an implementer to choose which metrics are implemented per protocol

8. Transactions Protocol Client Server Successful Responsiveness HTTP Jim Amazon 1 6 sec. SAP/R3 Jane SAP 1 17 sec. HTTP Joe HR 0 - FTP Jim ietf 1 212 Kbps HTTP Joe HR 1 25 sec. RealVideo Joe CNN 1 100.0% HTTP Jane HR 1 5 sec. HTTP Jim CNN 1 2 sec.

9. Flow-Based Aggregation Protocol Client Server Availability Responsiveness HTTP Jim Amazon 100% 6 sec. SAP/R3 Jane SAP 100% 17 sec. HTTP Joe HR 50% 25 sec. FTP Jim IETF 100% 212 Kbps RealVideo Joe CNN 100% 100.0% HTTP Jane HR 100% 5 sec. HTTP Jim CNN 100% 2 sec. All transactions that share common protocol/server/client 3-tuples are aggregated together, resulting in a set of metrics for all such unique 3-tuples.

10. Client-Based Aggregation Protocol Client Availability Responsiveness HTTP Jim 100% 4 sec. SAP/R3 Jane 100% 17 sec. HTTP Joe 50% 25 sec. FTP Jim 100% 212 Kbps RealVideo Joe 100% 100.0% HTTP Jane 100% 5 sec. All transactions that share common protocol/client tuples are aggregated together, resulting in a set of metrics for all such unique tuples.

11. Server-Based Aggregation Protocol Server Availability Responsiveness HTTP Amazon 100% 6 sec. SAP/R3 SAP 100% 17 sec. HTTP HR 66% 15 sec. FTP IETF 100% 212 Kbps RealVideo CNN 100% 100.0% HTTP CNN 100% 2 sec. All transactions that share common protocol/server tuples are aggregated together, resulting in a set of metrics for all such unique tuples.

12. Protocol-Based Aggregation Protocol Availability Responsiveness HTTP 80% 9.5 sec. SAP/R3 100% 17 sec. FTP 100% 212 Kbps RealVideo 100% 100.0% All transactions that share a common protocol are aggregated together, resulting in a set of metrics for all such unique protocols.

13. Can’t Aggregate Different Protocols These transactions can be aggregated in several ways, providing statistical summaries - for example summarizing all HTTP transactions,or all HTTP transactions to the HR Server. Note that data from different protocols may not be summarized because: 1. The performance characteristics of different protocols differ widely enough to render statistical analysis meaningless. 2. The responsiveness metrics of different protocols may be different, making a statistical analysis impossible.

14. Aggregation Algorithms • TransactionCount • The total number of transactions during this period • SuccessfulTransactions • The total number of transactions that were successful

15. Aggregation Algorithms (cont.) • ResponsivenessMean • The average of the responsiveness metric for all aggregatedtransactions that completed successfully • ResponsivenessMin • The minimum responsiveness metric for all aggregated transactions that completed successfully • ResponsivenessMax • The maximum responsiveness metric for all aggregated transactions that completed successfully • ResponsivenessBx • The count of successful transactions whose responsiveness metric fell into the range specified for Bx. There are 7 buckets specified. Because the performance of different protocols varies widely, the bucket ranges are specified separately for each protocol (in the protocolDirExtTable) so that they may be tuned to typical performance of each protocol.

16. APM Report Table APMReportControlTable Old APMReport APMReport APMReport APMReport APMReport APMReport Thu Mar 30 12-1PM Joe CNN WEB stats Jan POP POP3 stats Jan POP SMTP stats Bob HR PSOFT stats ... ... New

17. apmReportControlTable apmReportControlIndex Integer32, apmReportControlDataSource DataSource, apmReportControlAggregationType INTEGER, apmReportControlInterval Integer32, apmReportControlRequestedSize Integer32, apmReportControlGrantedSize Integer32, apmReportControlRequestedReports Integer32, apmReportControlGrantedReports Integer32, apmReportControlStartTime TimeStamp, apmReportControlReportNumber Integer32, apmReportControlInsertsDenied Integer32, apmReportControlDroppedFrames Counter32, apmReportControlOwner OwnerString, apmReportControlStatus RowStatus

18. apmReportControlTable apmReportControlAggregationType OBJECT-TYPE SYNTAX INTEGER { flows(1), -- Least Aggregation clients(2), servers(3), protocols(4) -- Most Aggregation }

19. apmReportTable apmReportIndex Integer32, apmReportServerAddress OCTET STRING, apmReportClientAddress OCTET STRING, apmReportTransactionType INTEGER, apmReportTransactionCount Integer32, apmReportSuccessfulTransactions Integer32, apmReportResponsivenessMean Integer32, apmReportResponsivenessMin Integer32, apmReportResponsivenessMax Integer32, apmReportResponsivenessB1 Integer32, apmReportResponsivenessB2 Integer32, apmReportResponsivenessB3 Integer32, apmReportResponsivenessB4 Integer32, apmReportResponsivenessB5 Integer32, apmReportResponsivenessB6 Integer32, apmReportResponsivenessB7 Integer32

20. apmReportTable Indexing • Indexes • apmReportControlIndex • apmReportIndex, • protocolDirLocalIndex • protocolDirLocalIndex, • apmReportServerAddress • apmReportClientAddress • apmReportResponsivenessType • An example of the indexing of this entry is 3.15.34.18.4.128.2.6.7.4.128.2.6.6.1

21. Protocol Directory Extensions protocolDirExtApmConfig INTEGER, protocolDirExtApmResponsivenessType INTEGER, protocolDirExtApmResponsivenessBoundary1 Integer32, protocolDirExtApmResponsivenessBoundary2 Integer32, protocolDirExtApmResponsivenessBoundary3 Integer32, protocolDirExtApmResponsivenessBoundary4 Integer32, protocolDirExtApmResponsivenessBoundary5 Integer32, protocolDirExtApmResponsivenessBoundary6 Integer32

22. ApmExceptionEntry • Extends the ProtocolDirectory • Multiple threshold entries possible per protocol ApmExceptionEntry ::= SEQUENCE { apmExceptionIndex Integer32, apmExceptionResponsivenessComparison INTEGER, apmExceptionResponsivenessThreshold Integer32, apmExceptionAvailabilityComparison INTEGER, apmExceptionAvailabilityThreshold Integer32, apmExceptionOwner OwnerString, apmExceptionStatus RowStatus }