IT Analytics and Big Data Making Your Life Easier

1. October 4, 2013 IT Analytics and Big Data Making Your Life Easier Paul Smith (Smitty) Service Management Architect

2. Data Overload Search, Predict, Optimize How can IT Analytics help? Agenda

3. Every business has 5-10 critical business process and applications. Slowdown or outage have a direct impact on their profits, revenue, customers and brand equity Trading halted for half a day on the biggest US exchange for financial options following an outage caused by software problems. Not surprisingly, many angry customers poured out their wrath via social networking after the largest video streaming company had a software outage for more than 20 hours A leading freight company lost $120 million in revenue because IT was unaware that critical warning messages were associated with their key freight delivery application.They were unable to deliver packages for an entire day due to downtime. Software problem led to two days of downtime at the largest bank in Europe has tarnished their image as the most reliable banking website. Airline canceled more than 700 flights and another 765 flights are delayed due to a software outage – Blamed ticketing partner while the real problem was on their end The Bottom Line:In Today’s World, the App can never go DOWN!!! 3

4. Relevant Operations Data is Huge A Typical Enterprise of 5000 servers with 125 applications across 2 or 3 data centers generates in excess of 1.4 TB of data per day Daily Metric Output: • 250 Mb of event data from 125,000 Events • 125Mb of endpoint mgmt data from 5K servers • 12 Gb of performance data for 5000 servers • 1 Gb of performance for 5000 Virtual Machine • 8 Gb or Application middleware data • Assumptions: 40% of servers running monitored middleware • Average 60 metrics each, collected every 15 min • Average PMDB insert 1000 bytes, 40 inserts/server • 500 Mb Application transaction tracking data for 125 Applications • 1 Tb Log file data per day • 200 Mb average per server (some will be smaller, some larger) • Example: WAS instances typically produce 400MB-750MB logs/day • .35Tb Security data collected per day • 9 Gb Storage Data per day: 175K fiber ports • 175 fiber ports,10 metrics per port, collected every 5 minutes, .5KB per port25K volumes, 10 metrics per volume, .5KB per volume5KB*(65K ports and volumes)*12*24 = 9.3 GB/day • 2Gb Network performance data for Data Center networks • 180x64 port Switches and 4 Routers to manage physical network. Data flow of approximately 1TB unstructured data, and .4TB metric data per day, Scaled to 20K servers, approx 4TB unstructured, 1.6TB metric data

5. Managing this much data requires Innovation Too Little: Limit Data Acquisition and risk missing important data Too Much: Flood IT Operations and risk missing important data Just Right? Today, we use Tools, Best Practices, Process, and Experience to get just the right amount of data Just Right: Analytics Solutions to examine all data, learn what is important, and escalate critical problems to Operations staff in a timely way. Just Right: Analytics Solutions to get to the heart of the problem. Just Right: Analytics Solutions to provide actionable insights. Too Much Data Overwhelms IT Not enough Data leads to Disaster

6. Enabling business transformation through IT Analytics Search Accelerate problem resolution through rapid analysis of structured and unstructured data.. Diagnose application and infrastructure issues with expert advice. Predict Enable predictive and preventative operations and application management with next generation behavioral learning analytics Specialized Capabilities Applications Workloads Systems Voice Mainframe Security Network Wireless Storage Assets Optimize Predict Search with ease all your data to do more with less Capabilities Optimize Optimize resource deployments with what-if and best fit planning tools. Track capacity and performance of applications Visibility Automation Control Plug & Play Architecture Integrated suite of capabilities leveraging existing Application Performance Management, Event Management and Monitoring Solutions Operational Environment It’s not just performance optimization. We also have to optimize with license cost and sub-capacity pricing in mind.

7. Predictive Outage Avoidance Ensure availability of applications and services Optimized Performance Track, Optimize, and Predict capacity and performance needs over time Faster Problem Resolution Find & correct problems faster with tools that determine actions required to resolve issues Improved Insight Enhance visibility into systems resource relationships while increasing customer satisfaction Perform Predict Know Resolve • Determine what resources are interdependent to assess impact of failures • Gain insight into what is important to your customer • Decrease customer churn and acquisition costs while increasing customer retention and satisfaction • Track capacity and performance of applications and services in classic and cloud environments • Optimize resource deployment with what-if and best fit planning tools • Increase utilization of existing assets • Use learning tools to augment custom best practices • Leverage statistical methods to maximize predictive warning • Use past maintenance to predict part failures • Identifyproblems quicker with insight to large unstructured repositories • Isolate problems quicker by bringing relevant unstructured data into problem investigations • Repair problems quicker with the right details quickly to hand. Enabling business transformation through IT Analytics Lower IT Administration Costs with Automated Analytics • Escalate performance and capacity issues automatically, reducing manual analysis efforts • Reduce manual customization using learning tools that automatically adjust to new normal • Detect and present problems with a proposed resolution, to be able to do more with less • Advice on Risk based automation to automate low risk tasks and escalate high risk fixes.

8. IT Operational Analytics Performance Data Unstructured Data Predictive Insights Log Analysis Avoid Outages and service degradation through early detection of abnormalities Improve insight though the analytical discover of metric relationships and trends Reduce root cause analysis by reducing time to isolate faulty components in complex infrastructure Identify problems quicker with insight to large unstructured repositories Isolate problems quicker by bringing relevant unstructured data into problem investigations Repair problems quicker with the right details quickly to hand. “by 2016, 20% of global 2000 enterprises will have IT operations analytics architectures in place...”- Gartner

9. Predictive Insights - The Problem • Why aren’t operations teams preventative today? • Too much data to analyze manually • Existing analytic techniques, such as standard thresholds, are not up to the task • They cannot detect problems while they are emerging (before business impact) • Set threshold too high, insufficient warning before total failure. • Set threshold too low, too much noise, everything is ignored If no there is no ‘early detection’ before the outage, operations teams can only react while outage is already in effect and already losing money...

10. Multivariate Analytics Statistical models can discover mathematical relationships between metrics Internet Banking Internet Banking A Application B C ESB Java / WAS D E AIX RHEL F G Oracle Core Banking Application H I Windows z/OS The extent this can be achieved depends on a number of factors, such as: range and type of data, availability of data, and stability of environment. Analytics falls back to a single metric if metrics are unrelated.

11. Example Scenario: Internet Banking Application Granger based analytics learns the mathematical relationship between metrics Web Response Time Internet Banking Anomaly Event Business Impacted User Requests A WRT Bad Web Response Time Web Response Time B C Typical Static Threshold WRT Good D E User Requests F G Time Early Warning • Learns ‘Web Response Time’ has a normal causal relationship with ‘User Requests’ - WRT gets slower as user load gets higher. • If this healthy historical relationship breaks down, say due to a memory leak, an anomaly is raised immediately • The problem is detected even while WRT service is “good” Leak H I Emerging problems can be detected even while service level are good in absolute term

12. Value Of The Watson Granger-based Analytic Approach • Learn normal operational behaviour across the infrastructure, including how metrics behave together. • Maximize Advance Warning: Identifies metric relationship changes that signal a problem long before traditional thresholds • Identify problems before you know to look for them • Detect service impacts that are not identifiable by fixed thresholds alone. • Assists with root cause analysis by indicating the most offending metrics. • Reduces expensive and time consuming false alerts. Provides a more intelligent real-time assessment of data, able to detect problems as they are emerging

13. Log Analysis – The Problem Find the right needle in the haystack – QUICKLY! It’s SLOW!! 404 ERROR Centralized, Distributed, Cloud, Resilient Architectures Increase Data Volume Everything is “green” Where do I start?? Logs, Traces,.. [10/9/12 5:51:38:295 GMT+05:30] 0000006a servlet E com.ibm.ws.webcontainer.servlet.ServletWrapper service SRVE0068E: Transactions Events Core files Metrics Config 010001100011100001110011000111110000110001 111111000110011100011

14. 14 14 Log Analysis – Key Capabilities Accelerate problem isolation, identification and repair Log Analysis • Advanced search and text analytics across large volumes of data • Index, search and analyze application, middleware, and infrastructure data • Quickly search and visualize application errors across thousands of log records • Cross index search across logs and documentation • Integrate log search with existing service management tooling to gain multiple perspectives on a specific instance of a problem

15. Analytics in IT - Capacity Management Definition from ITIL V3 • ITIL Capacity Management aims to ensure that the capacity of IT services and the IT infrastructure is able to deliver the agreed service level targets in a cost effective and timely manner. • Capacity Management considers all resources required to deliver the IT service, and plans for short, medium and long term business requirements. Sub Processes • Component Capacity Management • Service Capacity Management • Business Capacity Management • Capacity Management Reporting

16. Helps consolidate and reduce costs Reduces HW and labor costs Reduces number of physical servers required to run workloads Reduce number of required licenses Helps ensure application availability Are any resources overloaded? When will physical resources reach their limits? Have there been any significant changes in my environment between two weeks? Ensure supply can meet demand Ensure business policies are met Helps optimize resource utilization Right size virtual machines Identify trends for workload balancing Why Capacity Management is important

17. Use Analytics to Forecast • You already have the data! Use analytics to: • Forecast resource bottlenecks • Estimate impact of planned business change • Estimate impact of planned outage (ie maintenance) • Discover risky components • Discover hidden limits and potential unstable components • Give input to performance test decisions • Experiment with placement of workloads (cost, license, performance, etc)

18. Thank You!