It all Starts with the Right Questions

1. It all Starts with the Right Questions How to measure performance in a data-rich environment

2. Where to start? 2

3. Back to Basics Why do we measure? Determine the health of the organization Identify progress towards a defined end-state Provide objective evidence of improvement Meet compliance requirements How do we determine measures? Identified for us Easy to get the data Important for us to know Aligned with Strategy and/or Vision 3

4. Back to Basics How do we define success? Reputation Profitability Market growth How do we measure the intangibles? Intuition Metrics 4

5. Performance Measurement Systems 5

6. Define the measures Step 1 6

7. Basic Questions How are we performing against our financial projections? Profit, margin, or reserve attainment Sales or revenue attainment Cost management or budget attainment Cash Flow Earnings (EBT, EBIT, EBIDTA) Return (ROI, ROE) 7

8. Basic Questions How well are we meeting customer requirements as we deliver our product or service? How does that compare to our best competitor? Product or service outcomes such as on-time delivery, and quality or accuracy Customer engagement, loyalty, satisfaction, dissatisfaction Net Promoter Score, repurchase intent or repeat business Customer complaints or returns Market share or penetration 8

9. Basic Questions How effective are our key business processes*? Process outcomes such as yield, parts defective, error rate, capability indices How efficient are our key business processes? Process outcomes such as cycle time or throughput How are we improving our key business processes? Gains made in terms of cost savings, throughput improvement or improved process capability 9

10. Basic Questions How well are we keeping our workforce safe? OSHA required safety results How well are we preparing our workforce for current and future responsibilities? Indicators of training and development by position or certification/accreditation Indicators of career progression Indicators of succession planning or promotion from within How well are we meeting our workforce�s needs? Employee satisfaction and engagement results Grievances, absenteeism, turnover 10

11. Basic Questions How well are we making information available to people when they need it? Information systems process outcomes such as system outages Help desk results, or requests for new reports How well is the organization�s direction and goals understood by the workforce? Indicators from employee survey questions on understanding direction or goals Indicators related to performance management system alignment between personal goals and organizational goals How well are we achieving our strategy and action plans? Tracking indicators for accomplishment of action plans and strategies Indicators of favorable performance in an organizational level scorecard or dashboard 11

12. Decision Science Data How well are we keeping our workforce safe? OSHA How are we performing against our financial projections? GAAP Sarbanes-Oxley � 12

13. Balanced ScorecardNorton & Kaplan, 1996

14. Baldrige Criteria for Performance Excellence Framework

15. Early Focus How well are the organization�s direction and goals understood by the workforce? How well are we achieving our strategy and action plans? Business Intelligence Trending, Reporting, Performance Management How well are we meeting customer requirements as we deliver our product or service and how does that compare to our best competitor? Analytics Discrete Choice Modeling Why are things happening? What will happen next? 15

16. Later Focus How effective are our key business processes? How efficient are our key business processes? How well are we preparing our workforce for current and future responsibilities? How well are we making information available to people when they need it? 16

17. Improvement Focus How are we improving our key business processes? How well are we meeting our workforce�s needs? 17

18. Balance Impacts Decision-Making

19. But what we do can�t be measured! 19

20. Detail the Measures Step 2

21. Desired Definitions What does the measure mean? On-Time means on original request date, not rescheduled date Time-to-Hire starts with receipt of requisition and ends on the first day on the job Who owns the measure? Responsible or taking action as needed Where and how will the data be gathered? Data stream, frequency of measure, repository 21

22. Desired Definitions How will the data be manipulated or normalized for equivalent comparisons over time? Index (Sales/FTE or x hours of labor) How often will the measure be reported and analyzed? Daily, Weekly, Monthly, Quarterly Goals align with frequency of review 22

23. 23 Indicators Interrelate?

24. Reporting Measures Step 3

25. Useful Measures: Provide information to those who can act on it Available at the right frequency Available in the right format Are SMART Specific, measurable, achievable, relevant, time bound Achievable, but at what level of pain? Are aggregated and segmented Are continuously evaluated and updated 25

26. Evaluating Measures Where are we today? Level How does that compare to yesterday? Trend Is it any good? Comparison So what? Alignment/Integration with Vision, Strategy, Customer or Employee requirement 26

27. Taking Action Step 4

28. Acting on Analysis Celebrate the wins Identify course corrections as needed Create accountability for actions Toss out non-actionable indicators Identify better �right questions� Modify analysis techniques for �better answers� 28

29. �What Matters� is Changing Today Trend analysis and forecasting Standardized reporting Data visualization Analytics applied in business processes Simulations and scenario development Clustering and segmentation Regression analysis, discrete choice modeling In 18 Months Data visualization Simulations and scenario development Analytics applied in business processes Regression analysis, discrete choice modeling Trend analysis and forecasting Clustering and segmentation Standardized reporting 29

32. Resources 32 Baldrige Performance Excellence Program (2011). Criteria for Performance Excellence. Gaithersburg, MD: NIST. Davenport, T. H. & Harris, J. G. (2007). Competing on Analytics: The New Science of Winning. Boston: Harvard Business School Press. Hopkins, M. S. & Brokaw, L. (2011). Matchmaking with math: How analytics beats intuition to win customers. MITSloan Management Review, 52(2) pp. 35-41. Kaplan, R. S. & Norton, D. P. (1996) The Balanced Scorecard: Translating Strategy into Action. Boston: Harvard Business Press. Kaplan, R. S. & Norton, D. P. (2000) The Strategy-Focused Organization: How Balanced Scorecard Companies Thrive in the New Business Environment. Boston: Harvard Business Press. Kaplan, R. S. & Norton, D. P. (2004) Strategy Maps: Converting intangible Assets into Tangible Outcomes. Boston: Harvard Business Press. Kaplan, R. S. & Norton, D. P. (2006) Alignment: Using the Balanced Scorecard to Create Corporate Synergies. Boston: Harvard Business Press. Kaplan, R. S. & Norton, D. P. (2008) The Execution Premium: Linking Strategy to Operations for Competitive Advantage. Boston: Harvard Business Press. Kruschwitz, N. & Shockley, R. (2010). 10 data points: Information and analytics at work. MITSloan Management Review, 52(1) pp. 28-31. Oakes, D. (2008). Driven by Metrics. Quality Progress, September 2008. Stubbs, E. (2011). The Value of Business Analytics: Identifying the Path to Profitability. Hoboken, NJ: John Wiley.