Managing with Data Washington State Association of Counties (WSAC) June 6 th , 2013 Brian Willett, Performance Analyst

1. Managing with DataWashington State Association of Counties (WSAC)June 6th , 2013Brian Willett, Performance Analyst

2. My Premise Learning HOW to manage with data is just as important as learning WHAT to measure. Performance measures are a management TOOL, just like a knife or a screwdriver. The TOOL can be used for both GOOD and EVIL. INTENT and SKILL separate a home cook from a master chef from a serial killer.

3. The Yellow Bead Company Employees • Inspectors (1-2) • Can distinguish the color yellow • Can count up to “20” • Recorder (1) • Legible Penmanship • Can calculate the average of 5 whole numbers Supervisor (1) • A “natural leader” • Believes in holding people “accountable” • Able to “motivate” employees • Able to enforce strict adherence to procedure • Timer (1) • Can operate a digital stopwatch with basic training Willing Workers • Willing to “do their best” • Able to follow strict production procedures 3

4. The Yellow Bead Company Procedures Objective: “Produce” as many yellow beads as possible. Step 1 – Hold the bead box upside down, press the white button on the side, and empty the “trap”. Step 2 – Flip the bead box over and shake it exactly 5 times. Step 3 – Place the bead box on the table and press the white button on the side once. Step 4 – Flip the bead box upside down and show the production results to the inspectors . Step 5 – Inspectors independently count the number of yellow beads and tell the agreed upon number to the Recorder. The Recorder writes that number on the scorecard. Step 6 – Pass the bead box to the next Willing Worker and repeat the process four more times (Total of 5 times). Step 7 – After the 5th time through, pause work until told to resume and have the Recorder calculate the average production of the team for the quarter. Step 8 – The Recorder writes down how much time elapsed from start to finish. 4

5. The Yellow Bead Company debrief Anybody can achieve almost any goal by… …redefining the terms. …distorting or faking. …running up costs. “Numerical goals lead to distortion and faking, especially when the system is not capable of meeting the goal. Anybody will meet the quota (goal) allotted to them, especially if they are not responsible for the losses generated.” “Perhaps I can find new ways to motivate them?” Dr. W. Edwards Deming 5

6. It’s the process and system, not the people

7. What is a system? Outcomes: Organizational Benefits User Benefits Financial Stakeholder Benefits Suppliers Users & Brokers (Customers) Inputs Process Outputs

8. Foofaraw and Twaddle “All of the empowered, motivated, teamed-up, self-directed, incentivized, accountable, reengineered, and reinvented people you can muster cannot compensate for a dysfunctional system. When the system is functioning well, these other things are all just foofaraw. When the system is not functioning well, these things are still only empty, meaningless twaddle.” Peter Scholtes

9. What Does Change Look Like on a Run Chart? Process-Level Shift: Eight or more successive points on the same side of the median (baseline) Trend: Seven or more successive points in the same direction (increasing, decreasing, or staying exactly the same) Warning: Seeing a trend can depend on your sampling frequency Non-Random Variation Patterns: Look for repeating cycles and clusters Don’t look too hard for these – If they are present they tend to jump off the page 9

10. Analysis questions and management options Is there evidence of abnormal variation? (trend/process level shift) …If yes, put your finger on the abnormal variation and ask, “What happened here to cause it?” …If no, is the data capable of meeting or exceeding the target 100% of the time? …if no, you have two options: • Change the target so you can meet or exceed it 100% of the time, or… • Fundamentally change the underlying processes/systems.

11. Why not just worry about meeting targets? In this example, actual performance has never come in below the target, but the trend is stable and undesirable. If this were on a typical Red/Yellow/Green scorecard, it would be Green every time If you just base your evaluation on whether they hit the target or not in the past, you are missing what is likely to happen in the future. Better y Target Time

12. No time? No money?

13. Additional readings and references Understanding Data, T.S. Marshall, 1990. The New Economics, Dr. W. Edwards Deming, 1994. The Leader’s Handbook, Peter Scholtes, 1998. We Don’t Make Widgets, Ken Miller, 2006. Extreme Government Makeover, Ken Miller, 2011. A Performance Management Framework for State and Local Government, National Performance Management Advisory Commission, 2010.http://www.pmcommission.org

14. More Information? Help? Local Government Performance Center Website:http://www.sao.wa.gov/PerformanceCenter Twitter: www.twitter.com/LocalGovPerform • Brian Willett • Performance Analyst • (360) 725-9731 • Brian.Willett@sao.wa.gov • Larisa Benson • Director of Performance Audit • (360) 725-9720 • Larisa.Benson@sao.wa.gov