Tivoli Analytics

1. Tivoli Analytics Upstate NY TUG May 24 2011

2. Statement Of Direction And Intent • All statements of direction or intent are provided for planning purposes only and are subject to change or withdrawal without notice • This presentation represents our current goals and objectives • Our development teams are now studying the product source code • We are listening to customers’ views on the future directions of our solutions

3. Operations landscape: Increasing cost, volume & complexity Rising operational costs of systems and networking Maintenance costs: $8 spent for every $1 spent on new infrastructure* Explosion in volume of data and information Difficulty in deploying new applications and services Unpredictable workload characteristics * Source: IDC 2007 Plus virtualization & cloud bring increased dynamicity & change Challenge: Improve the assurance of physical and virtual environments across applications, systems, networks and storage. Service Assurance Analytics approach: Increase automation of monitoring and detect problems before they become service affecting Move from “reactive” management to fixing things before they break. Predictive analytics! “After we fix the problem, then we set another threshold” “What are realistic baselines? How can I reduce false alerts?” “I’d like 30 minutes warning to know when my user experience is going to deteriorate”

4. Key Customer Needs in Business Analytics Business Analytics and Optimization Strategyand Services Financial Performance and Strategy Management Budgeting and planning, financial consolidation, scorecarding and strategy management, financial analytics and related reporting capabilities to help simplify, structure, & automate dynamic and sustainable financial performance and strategy management practices Business Intelligence Advanced Analytics & Optimization Query, reporting, analysis, scorecards and dashboards to enable decision makers to easily find, analyze & shareinformation for decision making Data mining, predictive modeling, & other techniques to identify meaningful patterns to predict future events Analytics Applications Applications that package business analytics capabilities, data models, process workflows and reports to address a particular domain or business problem IBM Business Analytics Software that Addresses Key Customer Needs

5. Adaptive Monitoring Topology Correlation Predictive Modelling Self-learning Forecasting Anomaly Detection Dynamic Thresholding Forward Trending What is Predictive Analytics? Predictive Analytics enable IT organizations to move from reactive to proactive management of services, reducing outages and improving business performance. "Analytics leverage data in a particular functional process (or application) to enable context-specific insight that is actionable.“ - Gartner Move the sensing and alerting, and eventually actions to earlier and earlier • Advance warning of service impact, deterioration or outage • Realistic service baselines • Avoidance of expensive and time-consuming false alerts • Detection of service impacts that are not identified by fixed thresholds alone • Swifter diagnosis of certain events and patterns • Identification of the underlying root cause to implement fixes IDC study: Predictive analytics initiatives show an average ROI of145%,in comparison to89%for non-predictive analytics* * Source: “Predictive Analytics and ROI: Lessons from IDC’s Financial Impact Study” paper, Henry D. Morris 145% 89% “After we fix the problem, then we set another threshold” “I’d like 30 minutes warning to know when my user experience is going to deteriorate”

6. Monitoring and Analytics Approaches Where we’ve been (broadly deployed) • Monitoring physical resources in a single domain • Analysis of historical data • Static thresholds Eg Server,app,network resources Eg CPU utilization Eg Service QoS Eg Server resource Historical Eg Warehouse BI Real-time Eg streaming analytics • Where we are (available capability) • Monitoring service level health • Streaming analytics of network data • Dynamic baselining & linear forecasting • Multi-domain correlation

7. Monitoring and Analytics Approaches • Where we’re going (technology in lab) • Monitoring physical and logical resources • Real-time analysis of data in motion across server, network, storage, application • Advanced univariate, and multivariate predictive analysis What are the use cases? ….

8. Case 1: Identify anomalies that could not be found with single KPI thresholds 1 “DMZ” Business Data 1. Web server traffic stable 2. Application memory rising 3. I/O accesses stable 4. Application performance within normal range Identify emerging problem, even if all still “green” Memory leak! 3 Mainframe Security, Proxy Servers Web Servers 2 Database Servers 4 DNS, Caching Application Servers File/Print Servers LAN Servers Load Balancing Application Acceleration Identify anomalies you can’t find with single KPI thresholds

9. Case 2: Identify problems in resources not in your management domain 1,2,3 – Web servers response times • Previously uncorrelated Web servers response times become correlated with each other. • Problem in a downstream resource in a different management domain- SAN errors Green stars – Applications performance degradation • Gateway problems “DMZ” 2 1 Business Data Mainframe 3 Security, Proxy Servers Web Servers Database Servers DNS, Caching Application Servers File/Print Servers LAN Servers Load Balancing Application Acceleration Gateways Identify “outside” problems that will affect service

10. Statistical analysis State to Service Analytics Event KPI calculation & correlation to correlation to Services service definition analytics Event correlation & enrichment analytics Service to Infrastructure Correlation Analytics in Tivoli Today

11. Streaming analytics engine Continuous Ingestion Continuous Complex Analysis in Microseconds • Processes millions of events per second • Used in finance, manufacturing, law enforcement

12. Mainframe Applications Security Network Workloads Wireless Storage Infrastructure Voice Direction Web UIs for Operations Users Web UIs for Reporting Users BI DataMart ETL Reporting Analysis Cognos / (Tivoli Common Reporting) Predictive Analytics Historical Warehouse Asset Data Streaming Analytics Enrichment with Asset Data Historical Analytics / Pattern Detection SPSS Business Service Topology Data Load (ETL, Aggregation Bus …) Management Servers Analytics in the context of the Business Service, Agent-based Analytics, Metric and Event Collection, and Mediation Energy Systems

13. Tivoli’s Predictive Analytics Lifecycle Score the Metrics Import Patterns (e.g. Topologies, or Rules) Closed loop analysis Improve customer experience Prevent business impacting events Increased operational efficiency Alert & Resolve Discover Relevant Patterns Refine Collect Metrics and Events Multi Domain/Vendor

14. Analytics Landscape Techniques Scenarios Capacityplanning Historical Pattern Detection & Model Building Streaminganalytics Event to Event Correlation PolynomialRegression VM placement DataMining Threshold violationdetection LinearRegression (Adaptive)Model Building MultivariateAnomalyPrediction TransformRegression UnivariateAnomalyForecasting Decisiontrees GrangerModeling Naïve Bayes PredictionUsing GrangerCausality KPI AnomalyDetection Correlation Anomaly Detection Kohonenclustering DemographicClustering AssociationRule Mining Anomaly inferenceon unmanaged resources Logistic Regression Dependencybased Filtering Holt Winters Topology based analysis KPI calculations Topology Correlation Model Building………….Analysis & Scoring……………….Prediction

15. Methodology: Temporal Causal Modeling by Graphical Granger Modeling • Granger causality • First introduced by the Nobel prize winning economist, Clive Granger • Definition: a time series x is said to “Granger cause” another time series y, if and only if regressing for y in terms of both past values of y and x (1) is statically significantly better than that of regressing in terms of past values of y only (2) • Combination of Granger Causality and cutting-edge modeling techniques provides efficient and effective methodology for Granger causal modeling of a large number of time series variables

16. Demo Screenshots: Event received

17. Investigation

18. Adding in Pertinent Events

19. Service Assurance Analytics • Identify anomalous KPI behavior without any thresholds. • Leverage any existing managed data. • Provide multi-domain analysis to identify complex interactions • Leverage near real time streaming analytics to identify complex interactions and subtle emerging problems across domains • Warn users in advance of service impact, deterioration or outage. • Adaptive algorithms which can leverage historical data • Focus on usefulness of results, not on individual algorithms. • Add new algorithms over time, without requiring users to become analytics experts.

20. SNMP TIP Visualization Analytic Algorithm Application Streaming Analytic Engine Mediation Service Assurance Analytics • Leverages IBM Information Managementassets to field a state of the art solution • Highly scalable and resilient streaming analytics engine • Powerful analytics algorithms, combining multiple approaches, designed to leverage the analytics engine for extensive scalability • Highly flexible and scalable data mediation layer providing turn key integrations and easily extendable capabilities • TIP based, native visualization • SNMP and Netcool/Omnibus native predictive alerts 3rd Party Event Consoles 3rd Party Event Consoles