Why do we measure?

1. Why do we measure? 15-Jan-2005

2. Thou has ordered all things in measure, and number, and weight Wisdom 11:21 Why do we Measure

3. Definition • Measuring is the act of assigning numbers to properties or characteristics (from Oak Ridge Associated Universities) Why do we Measure

4. What do we Measure? • Step 1: Decide what we want to improve • Often this is not as obvious as it sounds • Consider the NTS … We are most interested in our preparedness to serve our agencies • We don’t measure preparedness though.. • We measure QNI, QTC, hours, etc. Why do we Measure

5. What do we Measure? • Can we measure it directly? • Sometimes direct measures are feasible • Often what we want to achieve can be difficult or impossible to measure: • Morale • Excitement • Interest • In these cases, we may need some proxy Why do we Measure

6. Why do we Measure? • We measure to: • Quantify a situation • Understand what affects things we see • Regulate • Help understand and improve performance • Validate a hypothesis Why do we Measure

7. Quantify a Situation • If we are having trouble with a path that used to work, we might … • Measure transmitted power • Check SWR • Review solar numbers • Our objective is to understand the situation so we can understand our ability to affect it. Why do we Measure

8. Understand what affectsthings we see • If our output power was low, we may look to SWR to see if foldback engaged • We may check emitted spectrum to be sure our capacity isn’t being eaten by spurs • We may measure voltages in PA to understand if we have a problem there Why do we Measure

9. Regulate and Improve • Every month our EC’s report drills and tests • The act of measuring helps encourage the various counties to maintain some level of practice and training • The results can help us understand opportunities for improvement • Still, a proxy for preparedness Why do we Measure

10. Validate a Hypothesis • You have probably seen the following video but didn’t realize it was about measurement • Want to validate a hypothesis but don’t have a direct measure • Analysis of the situation to identify a suitable proxy measure • Engage expert to help identify appropriate proxy Why do we Measure

11. Why do we Measure

12. Video Debrief • Did you notice what the knight was doing when the villagers interrupted him? • The knight required data to validate the hypothesis • No direct measure since the village apparently didn’t have a witch-o-meter • What did you think of the chain of logic used to determine what to measure? Why do we Measure

13. Some Rules • The more directly you can measure the better • While we can’t always measure exactly what we want, the video gives some insight into the pitfalls • Just because you don’t have a direct measure, don’t give up. Often a reasonable proxy can be discovered Why do we Measure

14. Some Rules (cont’d) • Define exactly how to collect the data for the indicator and how to make the computation. Then make sure everyone understands • This isn’t as easy as it sounds. Look at PSHR Why do we Measure

15. Some Rules (cont’d) • If you can’t flowchart the process you want to improve, a performance measure will offer little insight into what you should improve. • This is in line with understanding what you want to measure. • Sometimes we entirely miss the process Why do we Measure

16. Causality • Often, we want to understand relationships so we can understand causality • The existence of a relationship does not prove cause and effect • There may be other factors not studied Why do we Measure

17. You can’t improve what you don’t measure Robert F. Bruner Why do we Measure

18. Measure! • Measure wisely • Measure often • Understand the limitations • Understand changes Why do we Measure