What Gets Measured Gets Undone

1. What Gets Measured Gets Undone Dr. Jim Mirabella Director of Institutional Research Professor of Statistics Florida Community College - Jacksonville (FCCJ) J. Michael Adams Corporate Manager, Quality Services Florida Power & Light/ FPL Group, Inc. Florida Sterling Conference Wednesday, May 31, 2000 Disney World, Orlando, Florida

2. What’s getting “undone” ? • The desired outcome and supporting processes that the measure(s) intended to describe • Why???? • Poor measures drive poor performance!

3. Objectives of the workshop • Know good measurements from poor measurements • Distinguish the implications of good and poor measures on performance • Balance an array of measures that favorably tracks a process’s outcome for planning, assessing performance, and analysis • Assess attendees’ measures • Any other expectation?

4. “Why” Measure at All? • Measurement is the language of progress and comparison • Provides a sense of where we are AND where we are going (planning) • Can guide a steady advancement toward established goals (tracking) • Can identify goal shortfalls, or over-achievement (analysis) • Communicates to the work force what is important to the organization (behavioral)

5. Good measures are born from SMART goals • Specific • Measurable • Agreed upon • Realistic • Time-Bound

6. The Challenge of Measuring • Workers might perceive it as a threat • Workers might disregard organizational goals, customers, products and services • Workers might focus on obtaining favorable measurements • Measuring items with no influence on organizational success = waste of time • result is a bean-counting approach that focuses on irrelevant details • Expected to do something!

7. The Challenges- examples • “Apples and oranges” • Impact on comparisons, benchmarking and performance • turnover • consistency • defects vs. defectives • commutes (measured in time or miles) • school achievement

8. “Where” are these measures • Organizational • Process • Department/work unit • Individual • Exercise

9. Organization Process Department/ Unit Individual Your measuresJot down your various performance measures for you, your department, unit and organization. Indicate the purpose of the measure ( 1= planning, 2= tracking and analysis; 3: behavioral changes)

10. Quality Delivery Cycle Time Waste Cost Defects Satisfaction Complaints Financials Price “What” Do We Measure?

11. Measurement Examples • Operations-related measures • reliability • timeliness of delivery • order processing time • errors / defects • product lead time • inventory turnover • cost of quality • employee

12. Measurement Examples • Customer-related measures • customer satisfaction • customer complaints • customer retention • Financial measures • market share • sales per employee • return on assets • return on sales

13. Baldrige/ Sterling:Results Category Private, Education, Health Care 7.1: Customer Focus Results; Student Performance, Patient and other Customer Focus 7.2: Financial and Market; Student and Stakeholder Focused Results, 7.3: Human Resource Result; Budgetary and Financial Results, Staff and Work System 7.4: Supplier and Partner Results; Faculty and Staff Results 7.5: Organizational Effectiveness

14. Determining and Reviewing Measurements for Balance • Customer • Financial • Human Resources • Supplier/ Partner • Organizational Baldrige Results 7.0 supplier dept process

15. Desired Outcomes- Airlinesoverall organization balance scorecard • “Assumed Quality”- arrive safely • Loyal Customers • Market differentiation, key value attributes: • on-time arrival • baggage handling • customer complaints • Profitability and Market share • Satisfied stakeholders (shareholders, partners) • Employee growth and retention

16. Airline Measurement System • Individual measures for each flight process • Group measures for overall airline • If problems occur with a flight, who do you blame -- the flight crew or the airline? • What measures do you affiliate with the flight crew? • What measures do you affiliate with the airline? • What is important to you for a satisfactory flying experience?

17. Review your measures • Are the organization measures reflective of all Baldrige results items? • Do the measures reward favorable behaviors? • Is there alignment with the contributing departments and suppliers? • Are the overall results managed as an outcome of a process?

18. What’s the opportunity? • Many organizational measurement systems are too short, too rigid, or used like a strict teacher’s ruler ... to whack rather than to motivate • Need to replace these outdated measurement systems with more dynamic measurement system that motivates continuous improvement in customer satisfaction, flexibility, and productivity … SIMULTANEOUSLY

19. Mis-measurement Systems • Unless specifically tuned to flight plan, measurement systems may: • yield irrelevant / misleading information • provoke behavior not conducive to strategy • Traditional measures ignore requirements & perspectives of customers (internal/external) • Bottom-line measures (profitability) too late for mid-course correction / remedial action

20. Mis-measurement Systems • Many measurement systems overlook key non-financial performance indicators • Measures often used for punishment rather than to promote learning • Many measurement systems are inflexible and limited in what they can do

21. What Should a Measurement System Do? • Measures must link operations to strategic goals • departments should know how they contribute separately and together toward strategic mission • System has to integrate financial / non-financial info in a way usable by managers • managers need right info at right time • Measurement system’s real value lies in its ability to focus all business activities on customer requirements

22. What Should a Measurement System Do? • Measure what is important to the customers • Motivate operations to continually improve against customer expectations • Identify and eliminate waste -- of both time and resources • Help to accelerate organizational learning and build a consensus for change when customer expectations shift

23. Measurement Mismanagement In an effort to increase market share, a computer service bureau strategizes to improve the timeliness of its voicemail service. The goal is to provide new customer with service w/in 24 hours. To help speed the process, a program is developed to accept verbal phone orders. Most new customers were online w/in a day as promised, causing the company to celebrate their success. A later audit revealed 70% error rate in order entry, with 30% of customers disputing their bill and eventually canceling service..

24. Measurement Mismanagement A Hi-Tech company sets objective to become highly profitable by being a product leader. To measure the performance of its marketing and R&D functions, the number of new products developed is the most watched barometer. An internal review reveals that in a sample of 20 new product introductions, 80% were delivered over a month late, and significant waste piles up in production. Management cannot understand why the accountant’s ink is red – after all, their yardstick tells them they are developing new products at a record rate.

25. Beware of …….. • Seemingly Simple Measures • How satisfied are your customers? • What is your employee turnover? • What is your client retention rate? • Is the value of our service worth the price?

26. Beware of …….. Treating customer perceptions as objective measures • Customer satisfaction  product quality • Customer satisfaction is a complex phenomenon • A well-handled complaint results in higher customer satisfaction than does no complaint • More reasons for complaint  more dissatisfaction

27. Beware of …….. Non-specific measurements • Results are actionable • Hard to improve what you cannot assess Failing to measure adequately • Think BALANCED SCORECARD • Don’t give employees an outlet for gaming success • Identify all areas important to customers

28. Beware of …….. Using results incorrectly • Don’t tie results to employee pay unless employees can directly influence results • Don’t base employee pay on results that cannot be measured

29. Recap of Objectives and Expectations • Know good measurements from poor measurements • Distinguish the implications of good and poor measures on performance • Balance an array of measures that favorably tracks a process’s outcome for planning, assessing performance, and analysis • Assess attendees measures

30. Thank you!!!!!! Jim Mirabella Mike Adams