How to Leverage Metrics to Support ITIL Processes

1. How to Leverage Metrics to Support ITIL Processes August 2, 2006 2:00pm EDT, 11:00am PDT Speaker: George Spafford, Principal Consultant, Pepperweed Consulting

2. Housekeeping • Submitting questions to speaker • Submit question at any time by using “Ask a question” section located on lower left-hand side of your console. • Questions about presentation content will be answered during 10 minute Q&A session at end of webcast. • Technical difficulties? • Click on “Help” link • Use “Ask a question” interface

3. Main Presentation

4. Agenda • Overall metric selection considerations • That metric selections must have value • Metrics and Behavior • Potential metrics for each process This Powerpoint file will be shared and can be obtained by emailing either Jupitermedia (webcasts@jupitermedia.com) or myself (george.spafford@Pepperweed.com).

5. So, what is your goal?

6. IT Enables the Attainment of Objectives and Goals Note, functional areas have objectives that they must achieve in order to support the organization attain its goals

7. What are your constraints?

8. Management of an IT Organization Requires Teamwork, Planning & Oversight To achieve the goal requires a proper framework of metrics that recognizes that IT, customers and management may have different reporting requirements that must be recognized and met.

9. A Process • Is a series of tasks with a goal • “Why are we doing this?” • If there isn’t a goal, then you do not have a process • You can say the same for a company • Metrics are used to measure the health of the process • Movement towards the goal • Productivity is just that – movement towards a goal • Metrics can also be used to measure the overall health of IT … to an extent

10. Metrics & Deming • Dr. Deming is misquoted as saying “If you can’t measure it you can’t manage it” • He did believe in collecting and using data as part of decision making • He also cautioned as one of his Seven Deadly Sins to avoid basing decisions solely on “visible numbers” • Deming felt that not everything was visible or “knowable” and thus not measurable

11. Metrics Must Have Value • They must clearly support attainment of the goal • Do not generate metrics just because you can • Do not spend more collecting and creating the metrics than what they are worth • Don’t spend $20 to report on something that amounts to only $1 • They must be desired by management – beware of the 10-100 page management report!!! • Understand what they need • Keep it concise! 1-2 pages – not 5, 10, 20, etc. • Use a communication medium and format that works for them • Select metrics that capture impacts to the organization – not just IT • What is the value of the 99.99% availability metric by itself if the business lost $1 million during that 0.01% of downtime?

12. Metrics & Causality • Each metric only tells part of the story • Increasing the metrics judiciously may explain more (then again they may not) • A metric can often change for reasons other than what you suspect • Consider if it is possible to use several metrics and perhaps some qualitative/subjective factors that can validate observations if something is important enough • Use absolute measures and ratios • 100% of 1 is very different than 100% of 10,000 • Invest your time and resources where they matter!

13. “Tell me how you will measure me and I will tell you how I will behave.” -- Dr. Eliyahu Goldratt

14. What behavior do you need? During each stage of the process adoption lifecycle you may need to emphasize certain behaviors over others and that will affect the metrics you select.

15. Always ask “What negative behavior may this metric create?” You may want to measure a metric thinking it will drive positive behavior only to find the reverse!!!

16. ITIL is a Systemic Approach • The power of ITIL lies in its systemic integration of processes areas – not simply piecemeal adoption • Any single process in isolation will reach a level of diminishing returns • As Goldratt has taught us, to optimize the throughput of a system requires optimization of the system – not just one area • The metrics selected for use must reflect a systemic approach Service Support, Service Delivery and Security

17. The following metrics are for consideration and judicious selection…

18. Change Management • Number of changes submitted • Number of changes in process (meaning the backlog) • Number of changes rejected • Number of changes implemented • Number of emergency changes • Number of unauthorized changes • Number of changes that exceeded the allowed change window time • Number of failed changes that did not have a backout plan • Lost units of the goal associated with failed changes (“We lost $100,000 in sales due to the outage associated with that failed change.”) • Number of changes implemented on schedule • Number of SLAs breached due to a failed change • Number of changes that failed during installation • Number of changes that caused an incident • Number of changes that caused a problem • Customer Satisfaction • “Number” could be an actual count or a ratio. If it’s a ratio, get agreement on what is used to compute the ratio

19. Configuration Management • Number of inaccurate Configuration Items (CIs) • Where the production CI doesn’t match the CI record • Number of failed changes due to inaccurate CIs • Number of incidents caused by inaccurate CIs • Amount of unplanned work caused by inaccurate CIs • Number of unused licenses (could be + or -) • We worry about this because license management is one of the frequent large ROI components for doing Configuration Mgt • Number of unauthorized CIs (entered without a RFC) • Customer Satisfaction • “Number” could be an actual count or a ratio. If it’s a ratio, get agreement on what is used to compute the ratio

20. Release Management • Number of releases that conformed to the organization’s Release Policy • Number of releases implemented according to schedule • Number of releases implemented late • Number of unauthorized CIs in the Definitive Software Library (DSL) • Number of releases that were not tested according to plan • Number of emergency releases • Customer Satisfaction • “Number” could be an actual count or a ratio. If it’s a ratio, get agreement on what is used to compute the ratio

21. The Service Desk • Number of calls to the SD • Calls = Phone, FAX, email and WWW • Number of calls handed by agent • Number of service requests • Number of incidents • Number of calls handled within SLA targets • Number of calls handled that exceeded SLA targets • Number of tickets addressed during the first contact • Number of tickets escalated due to timing • Number of tickets escalated due to skills required • Average time the caller waits in queue • Customer Satisfaction • “Number” could be an actual count or a ratio. If it’s a ratio, get agreement on what is used to compute the ratio

22. Incident Management • Number of incidents • Number of incidents resolved within SLA targets for each level of priority • Number of incidents escalated to each level of support • Average time to resolve incidents by priority • Number of incidents incorrectly recorded (Priority, Category, Etc.) • Number of incidents incorrectly assigned to the wrong resources • Customer Satisfaction • “Number” could be an actual count or a ratio. If it’s a ratio, get agreement on what is used to compute the ratio

23. Problem Management • Number of problems • Number of known errors • Number of known errors resolved • Number of RFCs raised by Problem Management • Customer Satisfaction • “Number” could be an actual count or a ratio. If it’s a ratio, get agreement on what is used to compute the ratio

24. Service Level Management • Number of services covered by SLAs • Number of SLAs that do not have required Operating Level Agreements and/or Underpinning Contracts • Number of SLA breaches • Number of SLA targets at risk • Business impact of breaches • Number of Service Complaints • From Customers • From Users • Number of Service Reviews conducted • Number of Service Reviews past due • Number of Service Improvement Plans (SIPs) Opened • Number of open tasks from SIPs • Number of SIPs closed • Customer Satisfaction • “Number” could be an actual count or a ratio. If it’s a ratio, get agreement on what is used to compute the ratio

25. Availability Management • Service availability expressed using an agreed upon measure • Too basic: Availability = Uptime / Time Possible • What metric or combination would be meaningful to the organization? • Mean time to detect • Mean time to repair (MTTR) • Mean Time Between Service Incidents (MTBSI) • Business Impact of outages • Number of services where availability targets were met • Customer Satisfaction • “Number” could be an actual count or a ratio. If it’s a ratio, get agreement on what is used to compute the ratio

26. Capacity Management • Number of services with unknown capacity requirements • Unplanned capacity purchases • Accuracy of capacity plan • Capacity Purchases vs Budgeted Amounts • Number of CIs with performance monitoring • Customer Satisfaction • “Number” could be an actual count or a ratio. If it’s a ratio, get agreement on what is used to compute the ratio

27. IT Service Continuity Management • Number of services with a continuity plan • Number of services without a continuity plan • Number of continuity plans tested • Number of continuity plans not tested according to schedule (the backlog) • Number of open issues raised by testing • Number of plans deemed “at risk” • Number of plans deemed “ineffective” • Customer Satisfaction • “Number” could be an actual count or a ratio. If it’s a ratio, get agreement on what is used to compute the ratio

28. IT Financial Management • Actual expenses relative to budget • This can be broken down many ways • Number of services with a known costs • Number of services reviewed per schedule • Charge back • Amount of IT costs absorbed • Profitability (if applicable) • Number of Services with a model in testing • Number of Services with a model implemented • Customer Satisfaction • “Number” could be an actual count or a ratio. If it’s a ratio, get agreement on what is used to compute the ratio

29. Security Management • Number of security incidents opened by severity • Number of security incidents closed by security • Number of services that have had security reviews • Number of security reviews pending • Number of risks identified • Number of risks mitigated to an acceptable level • Customer Satisfaction • “Number” could be an actual count or a ratio. If it’s a ratio, get agreement on what is used to compute the ratio

30. Food For Thought • Understand objectives and goals • Understand what stakeholders need • Keep reports focused on what the reader needs • Periodically review the metrics to see if changes are needed • Only select and use metrics that matter

31. Additional Resources • There are metrics to be found in each process chapter of the OGC ITIL books • The “Planning to Implement Service Management” volume is especially rich with metrics for consideration • The itSMF’s new “Metrics for IT Service Management” is an excellent resource • The last two items provide a tremendous wealth of metrics for consideration

32. Thank you! George Spafford George.Spafford@Pepperweed.com http://www.pepperweed.com Daily News Subscription Information and Archive (it’s a free newsletter covering technology business, regulatory compliance, security and process improvement)http://www.spaffordconsulting.com/dailynews.html

33. Questions?

34. If you have any further questions, e-mail webcasts@jupitermedia.com Thank you for attending