Guide to the Expression of Uncertainty in Measurement

1. Guide to the Expression of Uncertainty in Measurement Keith D. McCroan MARLAP Uncertainty Workshop October 24, 2005 Stateline, Nevada

2. Introduction • Guide to the Expression of Uncertainty in Measurement was published by the International Organization for Standardization in 1993 in the name of 7 international organizations • Corrected and reprinted in 1995 • Usually referred to simply as the “GUM”

3. The Seven Sponsors • International Bureau of Weights and Measures (BIPM) • International Electrotechnical Commission (IEC) • International Federation of Clinical Chemistry (IFCC) • International Organization for Standardization (ISO) • International Union of Pure and Applied Chemistry (IUPAC) • International Union of Pure and Applied Physics (IUPAP) • International Organization of Legal Metrology (OIML)

4. Stated Purposes • Promote full information on how uncertainty statements are arrived at • Provide a basis for the international comparison of measurement results

5. Benefits • Much flexibility in the guidance • Provides a conceptual framework for evaluating and expressing uncertainty • Promotes the use of standard terminology and notation • All of us can speak and write the same language when we discuss uncertainty

6. What Is Measurement Uncertainty? • “parameter, associated with the result of a measurement, that characterizes the dispersion of the values that could reasonably be attributed to the measurand” – GUM, VIM • Examples: • A standard deviation (1 sigma) or a multiple of it (e.g., 2 or 3 sigma) • The half-width of an interval having a stated level of confidence

7. The Measurand • In any measurement, the measurand is defined as the “particular quantity subject to measurement” • For example, if you’re trying to determine the massic activity of 239Pu in a specified sample of soil as of a specified date and time, that is the measurand

8. Error of Measurement • In metrology the error of a measurement is the difference between the result and the actual value of the measurand • The error is treated as a random variable • With mean and standard deviation • True even for systematic error (discussed later)

9. Error vs. Uncertainty • In metrology, error is primarily a theoretical concept, because its value is unknowable • Uncertainty is a more practical concept • Evaluating uncertainty allows you to place a bound on the likely size of the error • It is a critical aspect of metrology • A measured value without some indication of its uncertainty is useless

10. Random & Systematic Errors • Error can be decomposed into random and systematic parts • The random error varies when a measurement is repeated under the same conditions (e.g., radiation counting) • The systematic error remains fixed when the measurement is repeated under the same conditions (e.g, error in a γ-ray emission probability)

11. Correcting for Systematic Error • If you know that a substantial systematic error exists and you can estimate its value, include a correction (additive) or correction factor (multiplicative) in the model to account for it • Remember: the correction term or factor itself has uncertainty • A small residual systematic error generally remains after all known corrections have been applied

12. Uncertainty • Uncertainty of measurement accounts for random error and systematic error • Does not account for blunders or other spurious errors, such as those caused by equipment failure • Spurious errors represent loss of statistical control of the measurement process

13. The Measurement Model • Usually the final result of a measurement is not measured directly, but is calculated from other measured quantities through a functional relationship • We’ll call this function a “measurement model” • The model might involve several equations, but we’ll follow the GUM and represent it abstractly as a single equation:

14. Example: Radiochemistry In radiochemistry, a simple model might look like where a denotes massic activity (the measurand), Cs the sample count, ts the sample count time, ε the detection efficiency, etc.

15. Input and Output Quantities • In the generic model Y = f(X1,…,XN), the measurand is denoted by Y • Also called the output quantity • The quantities X1,…,XN are called input quantities • The value of the output quantity (measurand) is calculated from the values of the input quantities using the measurement model

16. Input and Output Estimates • When one performs a measurement, one obtains estimated values x1,x2,…,xN for the input quantities X1,X2,…,XN • These estimated values may be called input estimates • One plugs input estimates into the model and calculates an estimated value for the output quantity • The calculated estimate may be called an output estimate

17. Propagation of Uncertainty • When a measurement model is used to estimate the value of the measurand, the uncertainty of the output estimate is usually obtained by mathematically combining the uncertainties of the input estimates • The mathematical operation of combining the uncertainties is called propagation of uncertainty

18. Standard Uncertainty • Before propagating uncertainties of input estimates, you must express them in comparable forms • The commonly used approach is to express each uncertainty in the form of an estimated standard deviation, called a standard uncertainty • The standard uncertainty of an input estimate xi is denoted by u(xi) • Radiochemists traditionally called this “one sigma” uncertainty

19. Combined Standard Uncertainty • The standard uncertainty of an output estimate obtained by uncertainty propagation is called the combined standard uncertainty • The combined standard uncertainty of the output estimate y is denoted by uc(y)

20. Methods for Uncertainty Evaluation • The GUM classifies methods of uncertainty evaluation (for input estimates) as either Type A or Type B • Type A: method of evaluation by statistical analysis of series of observations • Type B: method of evaluation by any means other than statistical analysis of series of observations • If it isn’t Type A, it’s Type B

21. Combining Uncertainties • All uncertainty components are treated alike for the purpose of uncertainty propagation • One does not distinguish between Type A uncertainties and Type B uncertainties when propagating them to obtain the combined standard uncertainty

22. Random & Systematic • Twenty years ago, it was common to call a Type A uncertainty a “random uncertainty” and a Type B uncertainty a “systematic uncertainty” • The GUM explicitly disparages those terms now • So avoid them • But recall that the terms “random error” and “systematic error” are still accepted (when referring to error, not uncertainty)

23. Examples: Type A • Make a series of observations of an input quantity Xi • Let xi be the arithmetic mean and let u(xi) be the experimental standard deviation of the mean (the “standard error” of the mean) • Least-squares regression can also be a Type A method • If there is a well-defined number of “degrees of freedom” (number of observations minus number of parameters estimated), it’s probably a Type A method of evaluation

24. Examples: Type B • Often a Type B evaluation involves estimating a bound, a, for the largest possible error in the estimate, xi, and dividing by an appropriate constant based on an assumed distribution for the error • For example, if you believe the true value lies within ±a of the estimated value, xi, but you know nothing more than that, assume a rectangular distribution, and divide a by to obtain u(xi) • Example: Uncertainty associated with rounding on a digital display

25. Rectangular Distribution xi− a xi+ a xi

26. Triangular Distribution • Sometimes you can estimate a bound, a, for the error, but you believe that values near xi are more likely than those farther away • In this case, you might assume a triangular distribution for the error • If so, you divide a by to obtain u(xi) • Example: Capacity of a pipette, with a specified nominal volume and tolerance

27. Triangular Distribution xi− a xi+ a xi

28. Imported Values • There are many other possible Type B methods • E.g., using the value and standard uncertainty of the half-life of a radionuclide published by NNDC • A calibration certificate for a standard might provide a confidence interval for the value with some specified level of confidence, such as 95 % • Assume a normal distribution and derive standard uncertainty from percentiles of that distribution (e.g., if the confidence level is 95 %, divide the half-width of the confidence interval by 1.96)

29. What about Counting Uncertainty? • Make a radiation counting measurement, where C counts are observed • Let xi = C and u(xi) = • What type of uncertainty evaluation is this: Type A or Type B? • This method of evaluation presumes Poisson counting statistics • Beware – Sometimes the distribution isn’t Poisson • Note – Counting uncertainty isn’t the total uncertainty

30. Correlations • An issue sometimes neglected in uncertainty evaluation is the fact that some input estimates may be correlated with each other • May either increase or decrease the uncertainty of the final result • One common example is the correlation that often exists between the parameters for a calibration curve fit by least squares

31. Notation for Correlations • If you know there is a correlation between two input estimates xi and xj, you should evaluate it and propagate it • Estimated correlation coefficient (a number between −1 and +1) is denoted by r(xi,xj) • The estimated covariance of xi and xj is denoted by u(xi,xj) • u(xi,xj) = r(xi,xj) × u(xi) × u(xj)

32. Uncertainty Propagation Formula • Most commonly used equations for uncertainty propagation are based on the general equation shown below, which the GUM calls the “law of propagation of uncertainty” • MARLAP prefers “uncertainty propagation formula”

33. Sensitivity Coefficients • The partial derivatives ∂f/∂xi that appear in the uncertainty propagation formula are called sensitivity coefficients • These derivatives are evaluated at the measured values of the input estimates • OK to approximate them – You don’t necessarily have to calculate them using formulas from calculus

34. Components of Uncertainty • The term component of uncertainty means several things, but one definition is explicit in the GUM • The component of the combined standard uncertainty, uc(y), generated by the standard uncertainty u(xi) is the product of the absolute value of the sensitivity coefficient ∂f/∂xi and u(xi), which may be denoted by ui(y)

35. Uncertainty Propagation • Uncertainty propagation formula is derived from a first-order Taylor-polynomial approximation of f • It is commonly used, but the approximation is not great in some situations (e.g., dividing one value by another value with a very large relative uncertainty)

36. Automatic Uncertainty Propagation • Many find the uncertainty propagation formula intimidating, but it is actually straightforward • Simple enough to be done automatically in most cases by software libraries • In the presenter’s opinion, uncertainty propagation is one of the easiest aspects of uncertainty evaluation • The hard part is understanding the measurement process well enough to recognize and evaluate uncertainties that ought to be propagated

37. Expanded Uncertainty • It is common to multiply the combined standard uncertainty, uc(y), by a factor, k, chosen so that the interval y ± kuc(y) has a specified high probability of containing the true value of the measurand • GUM calls product U = k×uc(y) an expanded uncertainty • Factor k is called a coverage factor (often k=2 or 3) • The probability that y ± U contains the true value is called the coverage probability, p

38. Summary of Steps • Define the measurand and construct the mathematical model of the measurement • Obtain estimates, xi, of the input quantities • Evaluate the standard uncertainties u(xi), by Type A or Type B methods, and evaluate the covariance u(xi,xj) for each pair of correlated input estimates xi and xj • Apply the model to evaluate the output estimate, y

39. Summary of Steps…Continued • Propagate the standard uncertainties u(xi) and covariances u(xi,xj) to obtain the combined standard uncertainty uc(y) • Optionally, multiply uc(y) by a coverage factor, k, to obtain an expanded uncertainty, U • Report the result, y, with either the combined standard uncertainty, uc(y), or the expanded uncertainty, U • Explain the uncertainty clearly

40. Questions?