Benchmarking IT: The Cost/Quality Balance

1. Benchmarking IT: The Cost/Quality Balance Chris Smith

2. Comparing IT Performance • While high level metrics are helpful and can be used to communicate the value of IT - remember • They are database averages from surveys and • Do not necessarily reflect your environment’s intricacies • A Benchmark provides a more accurate view of your spending & staffing levels as it compares your organization to a set of peers with similar • Service Levels • Environment complexity • Workload • Number of locations supported • Geographic location • Level of outsourcing

3. Devices • MIPS • Minutes • Volume • Users • Software • Hardware • Personnel • Occupancy • Transmission Cost Workload Specific Measures and Definitions Gartner collects data in line with its industry standard “Consensus Models” to ensure comparability “Workload” represents a service provided and is balanced with cost. To compare with actual spending, an organisation’s workload is multiplied by their peers’ average unit cost: The organisation’s PC Count 12,500 • Based on operational expense • Labour not depreciated x £900 Peers’ average cost per PC £11,250,000 Peers’ cost for supporting the organisation’s PCs

4. A Complexity / Workload Low High Key Principles of Benchmarking The diagram below serves to illustrate the fundamental flaw in the “database average” approach to benchmarking: Company B appears to spend more than Company A when viewed at the simplistic “cost per desktop” level. However, when a more contextual analysis is performed and the degree of complexity is factored in (the red line represents the extrapolated trend line from a Gartner benchmark database), then it becomes apparent that Company B is actually the more cost efficient, and Company A spends more than other organisations with the same degree of complexity. Cost per Desktop B “Cost difference”

5. Government Trends in IT Spend to Run, Grow and Transform the Business IT Spending to Run, Grow, and Transform the Business • The distribution of IT spending between run, grow and transform activities has been similar for the last 4 years • The proportion of spend across these three categories is very similar for National and Local government Data Source: Gartner IT Key Metrics Data 2010

6. What if You Are Outsourced? Determination of whether an overall deal is good requires not just a comparison to some high level spending numbers but also some measures across other dimensions: • Customer Satisfaction…measures here can provide indications of how the provider is doing and whether services meet the needs and priorities of the business. • Relationship Assessment…provides a view on the levels of trust and control in managing the deal. • Cost Assessment…in an open relationship provide insight into whether and how efficiencies in service delivery can be gained; across roles played by both service provider and service recipient • Business Alignment… focussing on innovation and developments is the deal delivering the business improvements expected. Is the deal generating the business outcomes expected? Is the service provider delivering the service outcomes expected, and is pricing still competitive? Are both operational and management staff happy with the day-to-day and medium-term "outcomes" being generated? Is the working relationship healthy? Is the service provider meeting the contractual requirements?

7. Changing the Conversation from IT Cost to IT Value • All CIOs must manage a diverse set of stakeholders • The toughest stakeholders are those who hold or control the IT budget directly and indirectly • Obtaining clarity over which stakeholders maintain this IT budget control is vital • IT is expected to play one of 4 specific and different roles in any organization • The CIO does not always/often/ ever choose the role IT is expected to play • Controlling the conversation about the value of IT, within its chosen role, is critical to the CIO’s survival and the strategic impact IT can bring to the business through innovation

8. CIO Scorecard Results at a Glance The overall performance of the IT organization at XYZ is rated better than the average organization. RUN GROW TRANSFORM Business IT = EFFICIENCY + - Overall (relative to peer) EFFECTIVENESS Better than average performance is found in the areas of Cost Containment and Infrastructure & Operations. Average performance is found in the areas of applications organization and enterprise architecture. An improvement area is found inbusiness process management. INNOVATION The effectiveness and innovation ratings for XYZ are slightly above average.

9. Peer groups are determined based upon the role that IT plays in the enterprise The database demonstrates clear differentiation in the average performance in the underlying maturity models for “Team Players” and “Grinders.” XYZ is compared to other “Grinder” organizations based upon current role and “Team Player” based upon aspiration for each CIO Scorecard evaluation component. How to Compare *Rating based upon 1 to 5 scoring with 5 as most mature

10. The Major Dimensions of IT Performance • There are a number of dimensions in which IS performance can be managed. These dimensions are inter-related and over-focus on one will adversely affect the others • Process • Value for Money • Alignment • Service Availability • Service Quality

11. 1 sec 4 sec 3 sec 2 sec 0 sec 98.0% 98.0% 99.5% 99.5% 98.5% 98.5% 99.0% 99.0% 100% 100% 1 sec 4 sec 3 sec 2 sec 0 sec 99.5% 98.0% 98.5% 99.0% 100% 4 sec 3 sec 2 sec 1 sec 0 sec 100% 250% 200% 150% 50% 99.5% 98.0% 98.5% 99.0% 100% 250% 200% 150% 100% 50% 99.5% 98.0% 98.5% 99.0% 100% 99.5% 98.0% 98.5% 99.0% 100% 4 days 3 days 2 days 1 day 0 days 1 day 4 days 3 days 2 days 0 days 1 day 4 days 3 days 2 days 0 days XYZ Actual: XYZ Target: 4 hrs 16 hrs 12 hrs 8 hrs 0 hrs 75% 0% 25% 50% 100% Peer Average: 4 hrs 16 hrs 12 hrs 8 hrs 0 hrs 30 sec 120 sec 90 sec 60 sec 0 sec Service Level Attainment – Sample Output Mainframe Availability Mainframe Response • XYZ availability and response service level are competitive with peer organisations • End-user-facing service levels, such as help desk first call resolution, tend to fall below average peer performance as well as XYZ targets UNIX Availability UNIX Response Wintel Availability Wintel Response Network Availability Apps. % of Time Plan App. % of Budget Plan Telecom. Availability Voicemail Availability Apps. S1 Restore MAC Timeframe Apps. S2 Restore Desktop S1 Restore First Call Resolution Desktop S2 Restore Mean Time to Answer

12. Client XYZ Peer Average Process Maturity – Sample Output 0 1 2 3 4 5 Asset Management • XYZ exceeds database average maturity levels in two thirds of the assessed disciplines. • IT governance and relationship management are areas of particular strength, suggesting a strong focus on business value. • In contrast, low maturity for asset management and business continuity may indicate risk for higher cost and greater downtime Bus. Continuity and DR Capacity Planning Change Management Configuration Management Database Administration Facilities Management Incident Management Inventory Management IT Financial Management Monitoring Performance Management Problem Management Release Management Security Management Service Desk Service Level Management Storage Management Task Management Relationship Management IT Governance

13. Business Alignment – Sample Output IMPORTANCE SATISFACTION • Management assigns the greatest satisfaction to reducing operating cost—a strong XYZ focus in years past • An increase in industry competition, however, has shifted focus from cost savings to competitive advantage, including improving time to market and enabling revenue growth 5 4 3 2 1 1 2 3 4 5 Faster Time to Market Competitive Advantage Enable Revenue Growth Reduce Operating Cost Improve Customer Support Reduce Bus. Disruptions Reduce Direct IT Cost Enhance Business Agility

14. IT Service Satisfaction – Sample Output • Consistent with Process maturity findings, line of business managers are concerned with XYZ’s preparedness in terms of disaster recovery • Managers gave Voice and Data communications their highest satisfaction rating—4.44

15. Help Desk Quality

16. Help Desk Productivity

17. Desktop Productivity

18. Benchmarking IT: The Cost/Quality Balance Chris Smith