Managing by Metrics

1. Managing by Metrics Valerie Rovine Sunflower Systems

2. Topics • Why Metrics are Important • Metrics as a Result of Disciplined Program/Project Management • Defining Quality Metrics

3. Why do We Need Metrics? • Provide performance targets and keep us focused • Are indicators of the health of the program, project or organization • Drive corrective action through consistent analysis • Provide quantifiable demonstration of success or opportunities for improvement

4. Program Management – A 6 Step Cycle • Define outcomes • Define key performance indicators (metrics) to measure success • Refine/implement processes/behaviors to drive outcomes • Collect data to support metrics • Analyze metrics • Take corrective action as needed

5. Step 1: Defining Outcomes • What is important to my program or organization? • How do I define success? • Examples • Excellent customer service • New development on time and within budget • Systems meet or exceed quality standards and customer expectations (minimal defects, requirements met)

6. Step 1: Defining Outcomes Outcome drivers: • Regulatory Requirements • Internal Policies • Organizational Goals/Strategic Plans • Audit Findings • Standards/Best Practices • Changes to Business Environment

7. Step 2: Translating Outcomes to Key Performance Indicators • How do I know I’m successful? • What does ‘Excellent Customer Service’ mean to me and my organization? • Examples • Efficient and effective resolution of help desk tickets • Good system performance • Positive customer feedback

8. Step 2: Translating Outcomes to Key Performance Indicators How do I quantify these indicators? • Efficient and effective resolution of help desk tickets • Percentage of tickets resolved in one business day or less to customer satisfaction • Average speed of ticket completion • First call resolution • Reopened tickets • Percentage of tickets aged 5+ business days • Good system performance • Percentage of system uptime • Average downtime per incident • Database wait time • Database CPU time • Positive customer feedback • Percentage of customers rating helpdesk service as good or excellent

9. Step 2: Translating Outcomes to Key Performance Indicators How do I define my targets? • Efficient and effective resolution of help desk tickets • Percentage of tickets resolved in one business day or less to customer satisfaction – 75% • First call resolution – 50% • Reopened tickets – < 5% • Percentage of tickets aged 5+ business days – < 20 % • Good system performance • Percentage of system uptime – 98% • Average downtime per incident – < 10 minutes • Database wait time - < 10 seconds • Database CPU time > 90% • Positive customer feedback • Percentage of customers rating helpdesk service as good or excellent – > 90%

10. Step 3: Driving Outcomes through Processes/Behaviors What do I need to do to achieve my outcomes and performance targets? • Implement ticket management system to document and manage customer problems • Provide adequate/on-going training for help desk personnel • Implement ‘ticket science’ procedures to identify themes and common problems • Document resolutions to common problems and push information to customers • Adjust staffing levels for peak and off times • Implement redundant processes to ensure system availability • Tune the database regularly to enhance system performance

11. Step 4: Collecting KPI Data What information do I need to effectively evaluate my performance? • Number of tickets processed through helpdesk • Customer satisfaction with resolution • Closure rate of helpdesk tickets • Aging information on helpdesk tickets • Actual uptime/downtime of system • System response times • Customer feedback on helpdesk performance

12. Step 4: Collecting KPI Data How can I collect this information with minimal disruption to my process? • Number of tickets processed through helpdesk • This metric was collected from ticket management system • Customer satisfaction with resolution • This metric was collected from ticket management system • Closure rate of helpdesk tickets • This metric was calculated using data collected from ticket management system • Aging information on helpdesk tickets • This metric was calculated using data collected from ticket management system • Actual uptime/downtime of system • This metric was collected through system monitoring software • Customer feedback on helpdesk performance • A survey was delivered automatically when a service ticket is closed • Also, a broader survey was conducted twice a year to solicit user feedback

13. Step 5: Analyzing Metrics • How do I know if I am on target for meeting my goals? • Regular data collection • Interpreting, analyzing and sharing data • Evaluations can be done from an Absolute or Relative perspective • Absolute: Does your metric indicate you have met your target outcome? • Relative: Does your metric indicate improvement over time?

14. Step 6: Taking Corrective Action • What do I do if I’m missing the mark? • Evaluate your processes and identify areas for improvement • Do I have adequate help desk coverage? Should I reevaluate the helpdesk staffing model? • Does my staff need more training on the system or on customer support protocol? • Do I need to identify alternate methods of support- community super users, self-service portal, web-based support? • Do I need more database resources? • Do I need to evaluate my hosting options?

15. Establishing Good Metrics • Does the metric tie to the organization’s goals? • Is the metric quantifiable? • Does the metric identify the expected result? • Does the metric take into account industry standards and benchmarks? • Is the metric attainable? • Were impacted individuals involved in defining the metric?

16. Considerations • What happens if my outcomes change? • How do I know I’m asking the right questions? • How can I ensure I’m collecting/calculating the right data?

17. Questions?